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CHAPTER 1 



Background and Introduction 



1-1. Introduction. This manual is coiiccrncJ with 
a process and a product The process is Instruc- 
tional System Dtivelopment (ISD) The product 
resulting from the apphcation of this process is an 
instructional system. 

a. Essentially, ISD is a systematic procedure for 
assuring application of mstructional technology to 
course planning and developmenL Thf planning 
concerns* the need for instruction, the facilities, the 
instructional equipment, the instructional proce- 
dures, and the personnel (learners, instructors, in- 
structional supervisors and managers, curriculum 
developers, media specialists, etc.). 

b. The resulting instructional system is an inte- 
grated combination of resources (students, in- 
structors, materials, equipment, and facilities), 
techniques, and procedures performing efficiently 
the functions required to achieve specified learn- 
ing objectives. ISD should provide mstniction 
specifically designed to teach people m minimum 
time and cost the knowledges and skills needed to 
perform a job. 

c. The ISD process, a synthesis of ideas and 
procedures from several sources, is a relatively 
recent development The management techniques 
were drawn from the planning systems devised to 
manage the development of complex weapon 
systems, the concepts of instructional design were 
drawn from programmed instruction, and develop- 
ments in instructional hardware make feasible to- 
day some instructional tactics that were not prac- 
tical a few years ago The Air Force developed 
its first major instructional system in 1965, and, 
by now, has had substantial experience in applying 
the process described in this manual. The applica- 
tions have involve4 a wide range of subjects, course 
lengths, and student populations for both new and 
existing courses. Because the application of ISD 



has consislcnll) produced gains in the cfficienc) 
of instruction. Air Force polic\ directs use of the 
ISD process. 

1-2. Air Force Policy: 

a Air Force policy regarding ISD is as follows. 
{ I ) Apply ISD to produce all new instruc- 
tional systems 

(2) Selectively apply ISU^o existing instruc- 
tional systems where economicalrv feasible. 

(3) The objective of ISD isjto train to re- 
quirements To achieve thjsr^each instructional 
system must be designed to contain only the edu- 
cation/training appropriate to the individual. 
Education /training is to be measured on pro- 
ficiency, not course length. 

(4) The intent in applying ISD is to develop 
quality training at the least cost. This is not 
necessarily the same as "cheap'' training. The 
distinction is most important. 

b. To further the orderly application of ISD 
for each command's instructional programs, each 
command will prepare a programming plan or 
regulation that outlines the objectives and re- 
sponsibilities for ISD. A copy of each plan/regula- 
tion will be forwarded to the Air Staff office of 
primary responsibility for ISD and be distributed 
as required. 

c Each major command submits a semiannual 
Instructional System Development Status Report, 
RCS. HAF-DPP (SA) 7303. The report covering 
the period 1 January through 30 June is due by 

30 July, and the report for the 1 July through 

31 December period is, due by 30 January. The 
report is submitted to AF/ DPPT. AF/'DPPT 
will distribute the report to additional agencies 
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VALIDATE INSTRUCTION 



•CONSTRAINTS- 



LEGEND CURRICULUM LOOP 



FEEDBACK AND INTERACTION LOOP 
figure 1-1. Model for Instructional System Development 



required. Refer to attachment 5 for an uutlme of 
the data required. 

1-3. Model for Instructional System 
Development: 

a* The Air Force uses the five step ISD model 
illustrated in figure 1-f ^ The process requires 

(1) determining precisely what are the hu- 
man performance requirements on the job. 

(2) determining the education/training re- 
quirements. This involves determining who is to 
do the job, and what education or training is 
necessary to enable them to do it in an acceptable 
way. 

(3) defining these instructional - needs as 
specific, behaviorally stated objectives and devis- 
ing test items to determine if and when the stu- 
dents atfain the objectives 

(4) designing instructional procedures and 
materials that will develop the skills and knowl- 
edges the students need to reach the objectives, 
and validating these procedures and materials to 
be sure they really do provide the needed in- 
struction — no more, no less. Whatever portion of 
thfe instruction- does not equip the students to 

There are other models containing fewer or more steps 
Despite this apparent difference, the process mvolved is 
basically the same. 

'In this manual, instritctton can be read as education or 
training 



meet the objectives (and ultimately, to do the 
job) must be redesigned until it works 

(5) conducting and evaluating the instruc- 
tion, and later, evaluating the students in terms 
of their ability to do the job. / 

b. The model illustrated in figure 1-1 preseribes 
an approach to the planning and development of 
instruction that is adaptive to all education and 
training programs. The model is applicable to re- 
vision and improvement of ongoing courses and 
instructional systems as well as to the design of 
new instructional systems. It is composed of log- 
ically interacting steps. The output of each step is 
intended to provide the input needed to accomplish 
a later step(s). As figure 1-1 shows, all parts of 
the mojel are interrelated. Changes which occur 
during one step of the model affect other steps. 
For*example, evaluation of the graduates' ability 
to do the job* might reveal that they cannot 
properly perform part of the job. This might be a 
result of improperly designed instruction — or it 
might be that the job has changed since the in- 
struction was implemented. In the first instance, 
modification of the instruction is indicated. In 
the latter instance, a return to Step 1 and a partial 
or even complete analysis of job requirements 
might have to be reaccomplished. 

c. Since there is feedback and mteraction be- 
tween steps of the model, sometimes portions of 
several steps can be accomplished simultaneously. 
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d The application of the ISD process md> re- 
quire compromises Limited funds frequently rule 
out desirable options when selecting training media 
Limited time available tu develop the instruc- 
tional system ma> necessitate a less than thorough 
analysis of system requirements, or using a smaller 
than desirable population for validation. These 
constraints must be considered in the design of the 
system. Of course, the goal is to devise an instruc- 
tional system that includes the best possible com- 
bination of alternatives, where "best" is defined as 
quality training in the most cost-effective way 
available. 

e. The following paragraphs provide a brief 
overview of the steps of the model for ISD 

(1) Analyze system requirements {Step h 
Before developing instruction, the instructional 
system designer must become familiar with everv 
aspect of the operational system, job. or educa- 
tional situation for which there is a presumed need 
for mstruction (In the beginning, there can only 
be the presumption of a need. Through applica- 

1^ tion of the ISD process, a determination is made 
as to whether the presumption is correct and, if 
it is, precisely what instruction is needed ) Analyz- 
ing system requirements primarily involves data 
collection, data on the system, its purpose, man's 
functional responsibilities in the system, the sub- 
systems and components that make up the system, 
support equipment or materials, and information 
on how the system operates, is maintained, or used 
The product of this step is a definitive list of all 
job tasks, the equipment or materials involved with 
each task, the conditions under which the tasks 
must be performed, and the work quality require- 
ments that must be met For later use, data is 
also needed on frequency of performance of the 
tasks, safety requirements, criticality, complexity, 
and the difficulty of learning or teaching Such 
determinations are made by observing the per- 
formance of personnel in the actual work environ- 
ment, by interviewing or surveying supervisors and 
subject matter specialists, and by studying the ap- 
plicat)le specialty description, training standard, 
related training documents, design documents, 
technical orders, and other documents which pro- 
vide information on the system. 

(2) Define education training requirements 
(Step 2)'^ 

(a) Based on job performance require- 
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ments, decisions are made eoncerning methods for 
acquiring qualified personnel. Instruction may not 
be required at all Training is costly, and if not 
needed should not be given The problem may 
not be lack of skills or knowledges, but rather 
motivation or working conditions Hiring or re- 
assigning people who can do the job may be more 
cost-effective than education or training. Perhaps 
the job could be redesigned, or job aids might be 
devised to reduce or eliminate training. 

(b) Once it is decided that training is re- 
quired, a determination is made of who Will be 
trained. Then the tasks defined during Step 1 are 
analyzed and compared with the skills, knowledge^, 
and abilities of the target population to determine 
what training is needed. Factors such as criticality, 
frequency of performance, complexity of the tasks, 
difficulty of learning, and time interval from train- 
ing until initial performance must be considered 
Other considerations are availability of time, avail- 
ability of instructors, availability of equipment and 
facilities, costs, and availability of funding. 

(3) De\elop objectives and tests (Step 3) 

(a) The instructional objectives are de- 
rived from the education or training requirements 
These objectives should be specific and stated m 
terms of the expected behavior or performance. 
They should clearly identify what the student is 
expected to do, the conditions under which he is 
expected to do it, and the acceptable standard of 
performance. Objectives should be expressed in 
terms that will ensure that the instructional intent 
IS communicated without ambiguity to all con- 
cerned. 

(b) Identifying specific behavioral objec- 
tives IS only a part of Step 3. The other part is 
designing test items to measure the attainment of 
these objectives. These are called criterion test 
items because they test against the criteria for suc- 
cessful performance that were identified in the 
objectives. On a criterion-referenced test, the stu- 
dents are rated on their ability to achieve the ob- 
jectives. How others score on the test has no 
bearing on an individual's grade. (This is in con- 
trast to the norm-referenced test in which the stu- 
dent is graded on how well he does in comparison 
with the other students.) 

(c) The criterion test items are used in 
several different ways. As already noted, they are 
used to determine whether or not students have 
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attained objectives When administered to the 
student at the beginning of a course or unit of in- 
struction, the test results can be used to determine 
which parts of the mstruction he can safel> by- 
pass. When administered to a group of people 
who are similar to those for whom the instruction 
is to be designed (the target population), the test 
results can be used as an aid to course, instruc- 
tional material design For example, if most of 
the people can handle the test item(s) covering 
a particular objeetivje, it may be that the objective 
should not be included m the course or the amount 
of instruction can be reduced 

(4) Plan, develop, and validate instruction 
(Step 4). The accomplishment of this step requires 
the careful planning of instruction lo satisfy the 
objectives This involves sequencing learning ac- 
tivities or exercises in an order' that produces the 
required learning in the' shortest time. It also in- 
volves .the ielection. and planning of instructional 
methods, media, and equipment which most effec- 
tively support learning objectives Accomplishment 
of these planning activities will provide the infor- 
mation needed to identify resource requirements. 
This step also involves the development and valida- 
tion of instructional materials. The validation proc- 
ess ensures that the instructional materials and 
process enable the student to achieve the stated 
objectives* This determination is made through a 
series of tryouts with students in which the appro- 
priate criterion test items serve as the measuring 
device The instruction that does not validate 
(work) during the tryouts must be modified until 
it does. 

(5) Conduct and evaluate instruction 
(Step 5}' 

(a) After the instruction has been devel- 
oped and validated, it is ready to be implemented 
Resources should be available Instructors, super- 
visors, and course managers should be familiar 
with the instructional system materials and their 
roles and function in the system. 

(b) Despite the constant efforts of every- 
one involved in its design, a newly implemented 
system seldom is completely successful Evalua- 
tion must occur continuously to ensure that any 
weaknesses are corrected until the system works as 
intended. This evaluation involves both the con- 
duct of the instruction and the performance on 
the job of the graduates It provides the feed- 



back necessary to maintain the quality and cost- 
effectiveness of an instructional system 

(c) Since ISD links training with actual 
job requirements, periodic change is to be expected 
in all instructional systems. The ISD process is a 
continuing c>cle which allows training personnel 
to be responsive to these changes as they occur. 
When a need for change or updating of the in- 
struction is indicated, the same ISD process is 
employed, 

1-4. Application of the ISD Process: 

a. In describing the ISD process, the emphasis 
has been on the component tasks. But there is also 
an accompanying logic and decision process. The 
process norfnally is initiated by the identification 
of a performance deficiency (;iew equipment com- 
ing into the inventory, production isn't up to 
standard, too many rejects, personal injuries, "they 
don't have the right attitude," etc.). After de- 
termining what the job involves (Step 1), the. in- 
structional system designer must determine whether 
the performance deficiency can be corrected by in- 
struction If It cannot, application of the ISD 
process is terminated. If instruction is the solu- 
tion, the designer must determine what instruction 
is needed to enable the target population to meet 
the job requirements and how best to meet this 

• need (Step 2). Although the tendency is to think 
immediately in terms of formal courses, the deci- 
sion process requires the consideration first of less 
expensive alternatives (assigning or reassigning 
qualified personnel, use of performance aids, etc.). 
If assigning qualified personnel resolves the proB- 
lem, that is as far as it is necessary to pursue the 
ISD process If a formal course is determined to 
be appropriate, there are still several options: use 
an existing course, modify an existing course, or de- 
velop a new course. The balance of the ISD process 
(Steps 3, 4, and 5) involves designing instruction 
to meet the identified needs in a cost-effective, 
manner. / 

b. All of the alfernatives identified above rep- 
resent application of the ISD process, though some 
alternatives will involve going through more steps 
of the model tha/others. Application of ISD 
will yield logical! and supportable decisions as to 
whether the proce^should be continued, or when 
it should be discontinued. 

c. It follows from what has been said that when 
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one begins to appl> the JSD process, one does not 
set out to "ISD a course'' or courses (Even if 
one were to begin b> attempting to ' appl> ISD 
to a course^ in the process, the job for which 
course graduates were destined would have to be 
considered. In so Joing, a possible conclusion 
might be that the course is not needed V Whether 
or not there will be a valid need for one or more 
courses will be determined by the findings as one 
proceeds to apply the process If this seemingly 
obvious concept is overlooked, much of the po- 
tential benefit from applying the process may be 
lost. 

d.There has been reference to an instructional sys- 
tem designer, as though one person performed the 
complete ISD process In some cases this may 



'During the transition period, as the Air Force imple- 
ments this process in all uf its Lourse de\eIopnncnt and 
revision, ISD may be applied to. a course as a result of 
a management decision rather than any known "per- 
formance deficiency " 



be what actuall) happen^ but generally applica- 
tion gf the ISD process involves the combined 
efforts of a number of people. These may include 
the subject matter, measurement, and media spe- 
cialists, and even personnel outside the organiza- 
tion such as occupational survey specialists from 
tht USAF Occupational Measurement Center ISD 
teams ma) be created to handle some of the large, 
complex projects. 



1-5, Summary: 

a. The ISD process provides a methodolgy for 
the orderly development or revision of Air Force 
education and training programs. When conscien- 
tiously applied, the process ensures cost-effective 
instruction that produces graduates capable of suc- 
cessful job performance. 

b. This chapter has provided an overview of the 
ISD process. Chapters 2 through 6 will each treat 
a step of this process in more detail. 
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CHAPTER 



Analysis of System Requirements ^ 



ANALYZE SYSTEM 
REQUIREMENTS 

A . 



-Nj' :OEFll!(E,EPUCAT(QH./ , 
VI TU^miKS REQUIREMENTS 



CONDUCT AND EVAIUATE 
: INSIBUCTION 




2-1. Introduction, The fiist i>tep in planning and 
developing an instructional system is the analysis 
of requirements of the operational system involved. 
This analysis is critical baause all other steps in 
the ISD model depend on the information provided 
by the analysis Jobs to be performed by quali- 
fied personnel are created by the requirements ol' 
these operational systems The analysts of opera- M 
tional sysjem requirements, or system analysis, is 
often thought to apply only to the design process for 
equipment oriented systems Within the context 
of ISD, the application is much broader. System 
analysis applies to administrative, equipment, man- 
agement, and support systems and subsystems, 
whether these systems are being developed, under- 
going modification, or already in operation. The 
concept of system analysis is also applicable to Air 
Force specialties and career areas This chapter 
describes the: 

a. concept of operational system analysis, 

b. sources of information for performing a sys- 
tem analysis; 

c. definition of job pertormance requirements; 
and 



d. sources and methods for collecting job re- 
quirements data, 

2-2. Concept of Operational System Analysis. 

a A system analysis performed for ISD pur- 
poses will foci^ upon the human component of the 
system (I)^ This analysis identifies the nature 
and scope of the human role in the system and the 
kinds of performance, or behavior, that will be 
required For example, it must be determined 
whether man is to serve as an operator, a main- 
tainer, a controller-manager or a combination of 
these. This determination cannot be made effec- 
livelv Without a comprehensive description of the 
individual characteristics of the system, career area, 
or Air Force specialty involved. 

b System analysis involves the collection of data , 
about the system and its operation. Analysis of 
this data should produce a description of the sys- 
tem in terms of: 



'The number in parenthesis indicates a referenced source 
for the information. The references are identified in 
attachment .2. 
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(1) mission or purpose, 

(2) functions or performance required to 
satisfy the mission, ' 

(3) major subsvsteniN and corn}X)ncrTts used 
to structure the system; 

(4) equipment or materials required to sup- 
port the system; 

(5) established concepts, policies, or pro- 
cedures for system operation, maintenance, or utili- 
zation; 

(6) effects ot environmental laclors on sys- 
tem operation and maintenance, and 

(7) the human's functional responsibilities 
in the system, that is. the human-system interface 

c The basis for planning and developing an in- 
structional system is the ta^k^ to be performed in 
operating, maintaining, controlling, or supporting 
the operational sv>teni Therutore. the functional 
responsibilities of the human component of the 
system forms the starting point for identifying job 
performance requirements Job performance re- 
quirements are described in terms of the job tasks, 
the conditions under which the tasks must be per- 
formed, and the work quality requirements that 
must be met In later steps of the ISD process, 
these tasks will be tmal>zed to determine the spe- 
cific behaviors (knowledges, skills, or attitudes) 
required for successful job accomplishment 

d. In addition to providing the information de- 
scribed in the preceding paragraphs, system analy- 
sis provides information needed to: ' . 

^ (1) determine whether or not a requirement 
to provide instruction exists; 

(2) identify requirements for additional in- 
structional systems; and 

(3) determine the impact of changes ia,sys- 
tem mission, operation and maintenance concepts, 
or job performance requirements upon existing in-, 
structional systems 

2-3. Sources of information for System Analysis. 

The following paragraphs provide a brief descrip- 
tion\)f general sources of data or information which 
may be of value in establishing system require- 
ments (more specific identification of data sources 
is provided in AF!^ 50-58, V^olume 2, Handbook 
for Designe/s of Instructional Systems, Task 
Analysis)^ 



a Equipmin'i SYs^^M Planning ano En-' 

GINEERING DaTA ' 

( 1 ) These data provide system r^t^uiremCnts 
jntormation for equipment oriented systems, such 
as aircratt, missiles, or detection and warning sys- 
tems, which are • being' developed or are under- 
going majpr modification. These data are valuable 
but .sometimes are not available, are difficult to 
obtain or translate into Air Force terms, or are 
very costly toobtain^. 

(2) Planning data are documented under 
57-. 80-, and 100-series Air Force regulaUons and 
AFR 800-2, Program Mana^^ement. Among the 
major sources of planning data are 

(a) Required operational capability 

(ROC) 

(b) Ptogram management directive 

(PMD) 

(c) Program management plan (PMP) 

(d) Qualitative and quantitative personnel 
requirements inventory (QQPRl) 

(e) Test plans (developed in accordance 
with AFR 80-14, Test and Evaluation), 

(3) Engineering data are contained in a va- 
riety of documents, including: 

(a) Engineering change proposal (ECP). 

(b) Maintenance engineering analysis 

(MEA). 

(c) Preliminary design review (PDR). 

(d) Training equipment planning informa- 
tion (TRPl) 

fe) Technical orders (TOs) 

b Air Force and Command Regulatory 
AND Procedural Pubijcations: 

* ( 1 )j These publications are used in analyzing 
system requirements for career areas, Air Force 
specialties' (AFSs), "^nd equipment oriented sys- 
tems already in operation Regulatory and pro- 
cedural publications describe systems, career areas, 
and specialties in terms of their subsystems, func- 
tions, capabilities, and use in attaining specific Air 
Force goals'; identify system missions and establish 
the concepts and policies for their operation, main- 
tenance, and us<e; and identify objective's and pro- 
* 

^n order that the mstj*i»^tional system will not he out 
dated because of opcration/equipment/policy changes in 
the pcnoJ between the determination of system re- 
quirements and the implementation of* the completed 
instructional system, information on probable changes 
should be included in the data gathered 
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vide polic> guidance on the iiiiplcnientation ot spe- 
cific Air Force programs Personnel classification ' 
and assignment policies and specialtv descriptions 
may be of value in analv/ing the requirenients of 
career areas of AFSs. In addition, reports and 
sunxmanes of actions resulting troni compliance 
with regulations and manuals arc often of value 
in performing . s>stem anal>sis For example, re- 
ports obtained through the application of the Main- 
tenance- Management Program (AFM 66-1) per- 
mit comparative anal>sis of actual and planned 
human performance The reports also identifv the 
number and Air Force SpecialU Code (AFSC) 
of the personnel performing a task and the fre- 
quenc> of performance 

(2) Procedural |nibJlicaTio;is provide detailed , 
instructions concerning the <idnfinistration. opera- 
tion, and maintenance ut Air Force ^equipment, 
materiel, and programs The\ describe program 
eleijients arid svstem components and establish 
their relationship ThCv delineate human pertorm- 
ance requirement:, and prescribe niethods and pro- 
cedures to be employed Procedural pubjic.ttions 
can take several forms, such as Air Force tech- 
nical order.s,job performance aids, military- s'peoi- . 
fications and standard, governmental iigcncy*' 
publications, and commercial publications Some 
procedural information nia> a|so be obtained from 
Air Force and command manuafs 

C AODITIONAl Soi RCf S Ol- InFORMAFION 

(U Education and training organizations 
have access to additional sources' of information 
which improve theif abilitv to identify and analyze 
system requirements Among thesc^ sources are 
evaluation report^ and feedback. Reports and feed- . 
back information are of the most value when used 
in conjunction with mforniation obtained from the 
sources identified in the preceding paragraphs 

(2) Instructional evaluation repQrts sum- 
marize the results of evaluations conducted in ac- 
cordance with Air Force directives (2) and 'or 
command directiv^es and supplements Since satis- 
factory jo.b performance is the criterion for instruc- 
tional systems, evaluation reports reflect the ack- 
quacy of existing instruction in meeting the 
requirements of the operational system, Analysis 
of any reported deficiency or requests for addi- 
tional instruction may lead to a more precise under- 
standing of system requirements. 
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(3) Feedback on the ellectiveness ot the in- 
structional and .operational systems ean be ob- 
tained through field trips, eonterences, or corcc- 
.spondence between using organizations and the 
education training activity. It can al^o be ob- 
tained throughyanatysis of statistical imports such 
as incom mission rates, maintenance management 
data (AFM 66-lj, and accident rates. *An addi- 
tional source of feedhacR. for training of flight 
Lrewnienibers, is the standardization-evaluation 
check Fe^edback ^rt')tained by these means pro- 
vides information on how a 'system is being op- 
erated, rnanagcd. or maintained . However, the 
field trip has a distinct advantage over the other 
methods of obtaining feeOback. It permits analysis 
of the system in its operational environment, thus 
providing a more precise identification of system 
requirements and the related human performance 
inipli^itions 

2-4, Definition of Job Performance Requirments: 

a A vital aspect of instructional system plan- 
ning afid development is the accurate identification 
of the human tasks required to operate, maintain, 
control, or support the system Instructional sys- 
tems are effective only to the extent to which they 
qualify" personnel for successful job performance. 
The job performance requirements are the tasks 
lequired of the human component of the system, 
the conditions under which these tasks must be 
performed, and the quality standards for accept- 
able perfornianee. This definition of tasks, condi- 
tions, and performance standards provides a clear 
.specification of what is to be done, why, when, 
and where it is to be done, and to what extent 
or degree of perfection it is to be done (3). 

b An important question to ask about the per- 
formance of any )ob is Is it observable under real 
vyorld conditions? While the training should equip 
the student to perform under real world condi- 
tions, it may not be possible to observe the per- 
formance of the task under real world conditions 
as a part of the training design process, One such 
type of situation occurs during development of 
new equipment or procedures when the task never 
has been performed and cannot be performed until 
installation of the new equipment or procedures, 
A related problem is that in which the training 
situation cannot be identical to the actual Job per- 
formance conditions. An example of this would 
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be training in performance ut hattlchcid first aid, 
the training is not performed under the stress con- 
ditions of the battlefield When the trainmg situa- 
tion cannot be identical to actual job performance, 
some approximation of job performance will be re- 
quired. 

c. Job performance lequirements establish the 
basis upon which instruction is designed and also 
provide the frame of ret^^^qice fcnr-evaluating the 
instruction (4).* In addition, >olypcrfp_rniance re- 
quirements are used in determining* the number 
of personnel and AFSs required; the interaction 
required between system personnel; work loca- 
tion; and the equipment, tools, or materials re- 
quired to accomplish the job Analysis of this in- 
formation makes possible the: 

(1) derivation of qualitative and quantita- 
tive education or training requirements, includmg 
instructional resources implications, 

(2) translation of dutv and task data into in- 
^ structional standards, learnmg objectives, and cri- 
terion tests; and 

(3) continuous updating of job requirement 

data. 

2-5. Sources and Methods for Collecting 
Job Requirements Data: 

, a. For New Weapon SuppoKr Systems. 

( ! ) Systems managed under AFR 800r2 per^ 
mit the acqiiistion of contractor-prepared data^ to 
supplement information already available from 
within the Air Force Some of these data concern 
planned use of the new equipment, some provide 
engineering data, some identify human perform- 
ance requirements, and some provide results of 
tests conducted underiAFR 80-14. 

(2) The various components of a complex 
weapon/support system are usually not in the 
. same stage of development at the same time Thus, 
while there may be much information available on 
some subsystems of the weapon/support system, 
there may be other subsystems developing more 
slowly for which very little information is avail- 
able. This situation presents a problem for the 
^instructional system designer since he may have 

'Data arc procured in accordance with procedures ,out- 
lined in AFR 310-1, Mana^i^emtnt of Controttor Data 



to make instructional planning decisions before 
all of the desired information is available. Such 
situations have existed in the past, and will prob- 
abl> arise in the future ISD dues not cause nor 
can it prevent this. But, to the extent that data 
have been gathered and analyzed in applying the 
ISD process at any given time, the instructional 
system designer has a more solid basis for making 
instructional planning decisions. As the weapon/ 
support system develops and more of the design 
becomes firm, more information becomes avail- 
able which can be used to update the ISD docu- 
mentation in anticipation of the next opportunity 
to update instructional planning decisions 

(3) Another special problerfi in planning for 
a new weapon/ support system is the need ttD^^lan 
concurrently for several versions of the training 
program As a new weapon support system is de- 
veloped, it progresses through several stages, such 
as early testing and evaluation of prototypes, im- 
plementation of the production model, and routine 
support once the equipment becomes operational. 
Each stage may require training support. How- 
ever, the training support needed probably will be 
somewhat different for each stage. There will be 
modifications in the hardware, and differences in 
the skill and experience of the people to be trained 
and the number requiring training. These differ- 
ences will alter the training requirements. Con- 
current planning of all this training (to the extent 
of determining training requirements, making de- 
cisions about methods for acquiring qualified per- 
sonnel, defining the instructional objectives, and 
making a preliminary identification gf resource re- 
quirements) IS necessary to assure minimal train- 
ing equipment costs and fullest use of the training 
equipment that is purchased. 

b. For New Training Programs Other 
THfAN weapon/Support Systems. There are 
some system development and modification pro- 
grams which do not employ system management 
procedures. Such systems may be either hard- 
ware or nonhardware oriented. In these instances, 
job performance requirements information in the 
form of published data is ilsually not readily avail- 
able. Nevertheless, information concerning the 
work to be done and the nature of the work is 
essential to ISD. Obtaining job requirements in- 
formation for such systems usually involves extern 
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sive research of engineering data and/'or pro- 
cedural publications. In addition to conducting re- 
search, it is often necessar> to consuh with the 
designers, engineers, and technicians v,ho planned 
and developed the s>stem. Every effort must be 
made to ensure the identification and documenta- 
tion of job requirements data. 

c. For Revision of Existing Instructional 
Programs: 

(1 ) The best source of information concern- 
ing job performance requirements for systems and 
career areas already in the operational inventory 
is the experienced person on the job. For profes- 

' sional military education, the person on the job 
may be the only source of information for deter- 
mining curriculum requirements. 

(2) Several methods are normally available 
for use in gathering the information needed to de- 
velop an inventory of the job tasks. They are de- 
scribed in the following paragraphs 

(a) Occupational Survey: 

The occupational survey is the Air 
Force procedure for the identification of the duties 
and tasks which comprise one or more shredouts, 
prefixes, specialties, career field lad%rs, or utiliza- 
tion fields; and for the collection, collation, and 
analysis of information concerning such duties and 
tasks. Through the use of job inventories, occupa- 
tional surveys obtain, from a representative sample 
of job incumbents., background information about 
them and specific data concerning the duties and 
tasks they actually perform This information is 
tabulated and computer analyzed with the results 
prepared iil the form of an occupational survey 
report. Data are reported in terms of percent per- 
forming tasks; relative time spent on tasks; and 
relative difficulty of tasks. 

^ Occupational surveys for the Air 
Force are conducted by the USAF Occupational 
Measurement Center, Lackland AFB, Texas. De- 
tailed information concerning procedures for re- 
questing surveys to be conducted, and for obtain- 
ing existing survey reports, are contained in AFM 
35-2 (5). (A sample portion of an occupational 
survey is included in chapter 9.) 

(b) Questionnaire. The questionnaire is a 
method of obtaining job requirements information 

ERLC 
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b> mail survey."* It requests the job incumbent to 
provide certain background information and de- 
scribe the job in his own words. When describing 
his job, the individual is asked to identify the duties 
and tasks he actually performs and to list the ma- 
terials, tools, and/ or equipment used. Incumbents 
are requested to work independently when com- 
pleting the questionnaire. However, supervisors 
may be asked to verify responses before the ques- • 
tionnaire is returned to the originator. This pro- 
cedure is designed to add comprehensiveness, ob- 
jectivity, and validity to t|ie information presented. 
Perhaps the chief economic advantages of this sur- 
vey method are that no structured forms are used, 
and the mail can be used as the means of com- 
munication. A disadvantage is the difficulty in cor- 
relating the relatively unstructured, handwritten 
information. Another disadvantage is that the ques- 
tionnaire method places a heavy demand on recall 
by those completing it. Incomplete forms increase ' 
the possibility of sampling errors. A well-designed 
questionnaire can minimize these disadvantages. 
Because know-how is needed to produce such a 
questionnaire, anyone planning to prepare a ques- 
tionnaire should first obtain guidance on how to do 
it. (Additional guidance on" the use of question- 
naires is provided in chapter 6 ) 

(c) Checklists: 

This survey method differs from the 
questionnaire method in that the checklist contains" 
a listing of duties and tasks believed to describe 
the , specialty or a postion within a specialty. The 
duties and tasks listed arc based upon information 
obtained from specialty descriptions, instructional 
standards, job proficiency guides, and procedural 
or technical publications. In this type of survey, 
the individual is a^ked to identify, usually by means 
of a the duties and tasks he performs. The 
results of the survey are compiled to form an up-, 
dated duty/task list for the specialty or position. 
The use of the checklist makes possible group 
administration to large samples of incumbents, thus 
making job requirements data available rapidly and 
economically from widel> representative [Popula- 
tions. The responses to the checklist are adapt- 

*Tf a survey of job incumbents is to be accomplished by 
mailed questionnaire or checklist, the questionnaire or 
checklist must be assigned a survey control number 
(SCN) by AF/DPP (see AFR 178-9, The Air Force 
Military Survey Program). 
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able to machine tabulation and the application of 
statistical analysis techniques. 

J_ There are a number of problems 
inherent in the checkhst method of conductmg 
surveys. One is the difficulty of constructing check- 
list items which clearly communicate to the in- 
cumbent the exact nature of the duty and task as- 
signments. Another is the ineffectiveness of the 
checklist for obtaining information relative to the 
sequence of lasL performance or the relationship 
among tasks. There also is a possibility that the 
checklist may not include all of the duties and 
tasks. 

(d) Individual Interview. In using this 
method for collecting job requirements data, a 
number of representative workers are selected fur 
interview.^ Those selected are interviewed con- 
cerning the duties and tasks the> perform. Inter- 
views are conducted on an individual basis, usually 
away froijn the work situation, and the informa- 
tion obtained is recorded on standardized forms. 
When all/ the interviews are completed the infor- 
mation is consolidated into a single job perform- 
ance requirements listing. One of the advantages 
of the individual interview method is that it usually 
yiekJs a better quality of information than the ques- 
tionnaire or the checklist. Among its disadvantages 
is the time required to conduct the interviews and 
compile job requirements data. 

/ Interview procedures should be de- 
termined prior to conducing the interview, and 
should be as explicit and detailed as procedures 
used to obtain data using a questionnaire. Areas 
for consideration include establishing contact with 
job incumbents, establishing rapport, outlining pur- 
poses of the interview, structuring the outlining 
of job duties deciding the level of behavior to be 
identified, and determining when to terminate the 
interview. (Additional guidance on conducting in- 
terviews IS provided ir^ chapter 6.) 

2 The itrferview method is especially 
valuable when u^'d in conjunction with the pre- 
viously mentioned checklist or questionnaire. When 
the questionnaire is based on numerous interviews 
and the results of the questionnaire are clarified 
by further interviews with subject matter special- 
ists, a valid inventory of job tasks is the result 

(e) Observation Interviews 

/ This method is similar to the indi- 



vidual interview except that it takes place at the 
w6rk site. While the worker performs his as- 
signed duties and tasks, the person gathering the 
information asks questions about the work being 
done and records the information As a result, 
the information obtained should be more specific, 
more complete, and more accurate than that ob- 
tained by other methods. Disadvantages are the 
relative slowness in obtaining the data, and the 
interview inevitably interferes with operational 
activities. The observation interview is more ap- 
propriately used to identif) the specific skills and 
teaching points involved in learning to perform a 
task, than for developing an inventory of job tasks 

^ A good job observation program will 
determine what the workers do. The questionnaire 
or interview will determine what they say they do 
The latter data should be used with some caution 
because there is a tendency for workers to say the> 
do what. they think they should do (6). 

d For Knowledges and Attitudes. This 
category applies primarily, but not solely, to 
academically oriented education courses and to 
programs for influencing attitudes. Many other 
programs also include cognitive (knowledges, per- 
ceptions, etc.) and affective (attitudes, feelings and 
emotions) objectives. 

( 1 ) Goal analysis: 

(a) Knowledges and attitudes are in the 
mind. They cannot be observed directly. The 
problem, tfien, is that none of the techniques pre- 
viously described (occupational survey, question- 
naire, checklist, or interview) will reliably indi- 
cate the knowledges and attitudes necessary to 
perform the job. But there is a viable alternative. 
goal analysis. All educational programs are based 
on broadly stated goals. (A goal, as the term is 
used here, is a general statement of educational 
purpose.) For example, job supervisors may indi- 
cate that the worker must "appreciate the impor- 
tance of ground safety in performing his job.*' This 
could become a goal for an instructional program. 
Before such a goal statement can be useful in the 
instructional design process; it must be translated 
into observable behaviors. The way this is done 
is to identify some observable behaviors that would 
be accepted by the supervisors as very likely indi- 
cations that the worker does indeed have this ap- 
preciation- (7). In other words, it is necessary to 
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find indicator behaviors and use these as highl> 
probable evidence that the nonobservable knowl- 
edges or attitudes exist Some behaviors that might 
be accepted as mdicator^* that the worker does '\ip- 
preciate the importance of ground safety in per- 
forming his job'' are: 

_/ Adheres to published safety proce- 
dures and regulations even when supervised. 

_2 Practices good Jhousekeeping in his 
work area. 

3^ Reports safety hazards. 

4^ Cautions others about their unsafe 

practices. 

^ Submits suggestions to improve safety. 

(b) These indicator behaviors y^ould then 
be treated as job performance requirements. (An 
application of goal analysis is illustrated in chap- 
ter 7) 

(2) Content analysis Some courses are in- 
tended to teach a bod> of knowledge (for example, 
astrophysics^ aptitude and achievement measure- 
ment, physiological psychology) The goal aim is 
to *'cover the subject," rather than to equip the 
student to deal with an> specific job applications, 
because the potential applications are so diverse 
and often unpredictable. In this case, too, the oc- 
cupational surveys, questionnaires, checkjists, or 
interviews described earlier in this chapter may not 
provide much useful guidance on what to include 
in the instruction. The alternative is to accept as 
a legitimate goal that the instruction should cover 
all of the subject matter identified in the title. If 
this IS done, the subject matter experts become 
the authorities on what the instruction should in- 
clude. What the subject matter experts provide 
would be the general instructional purpose state- 
ments (goals). These would serve as a substitute 
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tor the job performance requirement, and would 
become the starting point for instructional design 
Analysis of these broad content statements would 
lead to the definition of objectives and instructional 
content (8). 

e. Recording Job Performance Require- 
ments: 

( 1) A worksheet similar to the- one shown in 
figure 2-1 is both adaptable and recommended for 
the purpose of recording task information. (Other 
worksheet formats may be used.) When com- 
pleted, this^vorksheet becomes an inventory of the 
job performance requirements. 

(2) Guidance for the preparation and use of 
this form is provided in AFP 50-58, Volume II, 
and an example of its use is shown in AFP 50-58, 
Volume I, Handbook for Designers of Instructional 
Systems, Introduction. 

2-6. Summary, Actions relating to instructional 
system planning and development should be pre- 
ceded by a thorough analysis of the requirements 
of the system, career area, or specialty to which 
Its graduates will be assigned. Successful system 
analysis requires that all available sources of in- 
formation be employed. The data obtained from 
the various sources should be compared to ensure 
accurate identification and an adequate under- 
standing of requirements. System analysis must 
also provide the job performance requirements in* 
formation needed to determine if instruction is re- 
quired, or to develop or revise an instructional 
system. Instruction is effective only to the extent 
that it qualifies personnel for job performance. 
Therefore, an inventory of the duties and tasks to 
be performed by the human component of the sys- 
tem is the basis for instructional system develop- 
ment. Thorough analysis of system requirements 
will ensure that instructional systems are designed 
to meet operational or educational needs. 
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Figure 2-1. Example of a Worksheet for Recording Job fiequiremenfs 
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3-1. Introduction. The defining ul cducaX^on anci 
training requirements represents the first major 
decision-making step in planning and developing an 
instructional system The decisions to he made 
are: Is instruction necessary'^ If it is, who is to 
be taught? What instruction do they required? What 
method (s) should be used to provide the insthic-- 
tion? Preliminary decisions must also be made 
.about.. identification of resources required* to de- 
velop, implement, and support the instructional 
system. This chapter is concerned with the deci- 
sion-qiaking process as it relates to the. 

a. determination, analysis, and documentation 
of education and training requirements*; 

b. methods for acquiring qualified personnel; - 

c. criteria and constraints affecting the selec- 
tion of these methods; and 

d. preliminary identification of resource require- 
ments. 



3-2. Determination, Analysis, and Documenta- 
tion of Education/Training Requirements: 

a. in Step 2 the qualitative and quantitative 
education/training requirements are determined. 
The process of making these determinations begins 
with an analysis of job performance requirements 
identified during Step 1 of the ISD model. 

(1) The job performance requirements in- 
ventory that has been prepared must be compared' 
with what is known about the people who will have 
to perform the job. This comparison should indi- 
cate whether or not an instructional requirement 
exLsts If there is a valid requirement for instruc- 
tion, the comparison will also reveal for which 
tasks/portions of tasks instruction will be required. 

(2) If the workers are capable of meeting 
the job performance requirements, but do not do 
§0, the working conditions should be carefully ex- 
amined. There may be a number of '^possible 
causes: lack of motivation; human relations prob- 
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lems, pressures for overl> Jiigh productivit) , in- 
efficiently organized work area, insufficwnt light, 
uncomfortable temperature, etc. 

(3) If instruction is considered tor workers 
already on the job, the analysis should identif> 
those job performance requirements on which the> 
show a performance deficiency These will be the 
job performance requirements that will require in- 
struction; instruction will not be needed on the 
others The amount of the performance deficiencv 
will determine the amount of training needed 

(4) If mstruction is considered for workers 
who have no prior familiarity with the tasks, there 
still may be some job performance requirements 
that are so simple that instruction is not required, 

(5 ) That portion of the job performance re- 
quirements inventory for which instruction will be 
required must be analyzed to identify the associ- 
ated knowledges, skills, and their performance re- 
quirements If a worksheet similar to that shown 
in chapter 2, figure 2-1, was used to record job 
performance requirements information, the same 
worksheet can be used to develop the nntructional 
requirements information AFP 50-58, Volume If, 
provides the information on how to do this 

b The following paragraphs touch on some of 
the factors that should be considered^n determin- 
ing the instructional reqiiirements to support the 
job performance requirements and how these re- 
quirements should be met The factors that will 
be addressed are number performing the task, 
criticality of the task, frequency of performance, 
learning difficulty, time interval before initial per- 
formance, and prior experience. Although these 
factors are addressed separately, they must be con- 
sidered collectively Thus, a task that is rarely per- 
formed might still warrant tnstruction if it is a 
critical task (for example, mouth-to-mouth rcr 
suscitation) 

(1) Number performing the task. If a task 
is performed by a majority of the job incumbents, 
it probably is an instructional requirement. A task 
performed by only a small number of job incum- 
bents normally would be excluded When the task 
is performed by about half of the incumbents, the 
decision to include or exclude training on it is 
very dependent on th^ other factors Sometimes 
there is another alternative, defer the instruction 



until the individual is m the job situation, and then 
provide instruction for those who need it 

(2) Criticality of the task Criticality refers 
to the relative importance of a task to mission 
success as compared to the importance of other 
tasks In determining whether there is an instruc- 
tional requirement relative to a given task, critical- 
it> IS somelimes more important than the number 
performing For example, initially it might 'seem 
that a task performed b> a small number of job 
incumbehits should be excluded as an instructional 
requirement Yels further analysis may reveal that 
an inability to perform that task in a precise 
manner, when the need arises, could have a serious 
impact on the organization's mission capabilit) 
The ''best judgment" decision is made on the basis 
of both the number performing and the criticality 
of the task. ^ 

(3) Frequency of performance. In deciding 
what level of proficiency should be developed 
through formal instruction, the frequency of task 
performance ijs likely to have a direct bearing For 
example, the more often a task" is performed the 
greater the opportunity for the development of 
proficiency This would suggest limited initial in- 
struction followed by on-the-job training (OJT). 
In contravSt, iv ta'-k seldom performed but highly 
critical would likely require a high level of initial 
instruction. 

(4) Learning difficulty. Some tasks are re- 
latively easy to learn on the job and require little, 
if any, formal instruction. If analysis of the task 
reveals 'that it can be taught with little difficulty 
through OJT, the instructional requirement would 
be met by a combination of limited initial instruc- 
tion followed by OJT Conversely, a task difficult 
to teach on the job would require a high degree 
of formal instruction. 

(5) Time interval before initial performance. 
If analysi,s indicates there will be a substantial 
time interval between completion of formal in- 
struction and initial perforjiiance on the job, two 
decisions must be made; ' 

(a) Should formiil instruction be provided 
on the task(s), or should jthe instruction be de- 
ferred until the time when it will be needed? If, 
for example, certain electronic fundamentals are 
not required until the 5- or 7-skill level is attained. 
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then it IS questionable tu include those funda- 
mentals in a basic (3-skill level) course. 

(b) If the decision is made to provide 
formal instruction on the task(s). how much 
repetition will be necessarv to assure that the 
student retains the skill knowledge involved until 
it is needed on the |ob'^ 

These decisions have many significant implications 
including training costs and the proportionate 
distribution of training time and job productive 
time. 

(6) Prior experience Personnel selected for 
assignment within a specialty ma> possess some 
of the basic knowledges and skills needed for 
successful job performance. Criterion measures 
can be developed from an analysis of the task 
and knowledge statements derived from the job 
performance requirements. By administering sur- 
vey tests based on the criterion measures to a 
representative sample ot the target population, 
skills and knowledges already withm the capa- 
bilities of the students can be identified. It would 
be uneconomical to build into instruction the 
teaching of knowledges ur skills already possessed. 

3-3. Methods for Acquiring Qualified Personnel. 

Another decision to be made during this step ot 
the instructional system model concerns the se- 
lection of methods for acquiring personnel quali- 
fied to meet the job performance requirements 
Depending upon the kind and degree of quali- 
fication needed, any one or a combination of 
techniques may be employed. The following 
paragraphs describe methods which may be used. 

a. Selective Personnel Assignment. It may 
not be necessary to provide instruction to acquire 
personnel who possess the qualifications needed 
for job performance. Qualified personnel may be 
available as a result of system phaseout, or as a 
result of surpluses in allied career areas. For 
example, the Air Force phases out a computer 
system in favor of one of more advanced design 
and greater capability. The programmers assigned 
to the former system possess basic knowledges and 
skills which can be, and should be, utilized in 
the new system. In this example, instruction may 
or may not be required, or not as much may be 
needed. The job requirements of the Air Force 
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are constantly changing, theretore, previously ac- 
quired knowledges and skills should be considered 
and utilized wherever possible Another example 
of selective assignment is the recruitment of per- 
sonnel who possess needed qualifications. Air 
Force requirements for certain technical specialties 
and professional personnel, such as doctors, 
lawyers, and engineers are met in this manner. 

b ON-THt-JoB Trmning (OJT.), On-the-job 
training is a systematic application of self-study 
and the craftsman-apprentice principle (9). Under 
this program, specialty knowledge is acquired by 
enrolling in and studying courses designed for 
career development,^ while job proficiency is ac- 
quired by performing on the job under the super- 
vision of a skilled professional or technician. This 
combmation usually provides the qualifications 
needed after only a short period of experience. 
OJT is particularly suitable for developing job 
proficiency and is often used in conjunction with, 
or as a continuation of, instruction received through 
formal courses. OJT is frequently used as the 
method for retraining personnel from one career 
area (or specialty) to another (10). It also plays 
a major role in professional military education 
since most management training is accomplished 
through 6JT Therefore, OJT may be the most 
effective and economical method for satisfying 
education or training requirements 

c. Formal Coursls of Instruction. The Air 
Force requires a wide variety of specialties coni- 
posed of numerous combinations of knowledges 
and skills. Some jobs are highly complex and 
require long periods of instruction. Others are less 
complex and the instructional time required is 
correspondingly less. In some specialties em- 
phasis is placed on knowledge, while in others 
the development of skill is more import^t. To 
develop the behaviors (knowledges, skilllf and 
attitudes )^^ded for successful job performance, 
any one or a combination of the following al- 
ternatives may be used 

(1) Exisiins courses. Education or training 
requirements can often he satisfied through the 
use of existing courses that are available to the 
Air Force. These courses may be conducted by 
the Air Force, or provided to the Air Force by 

'Career Development Courses (CDCs) and/or courses 
prepared by the Extension Course Institute, 
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other governmental agencici), industr>, or civilian 
educational institutions (AFR 50-18, hiterser\ ue 
Formal School Training, requires a military service 
having a training requirement for ^hich no train- 
ing exists within that service to examine (he feasi- 
bility of using existing courses within the other 
services before establishmg a new course. (25)) 
Decisions relative to the suitability of these courses 
should be based on the requirements of the job(s). 
to be performed 

(2) Revised courses When it has been de- 
termined that an instructional requirement can- 
not be adequately met b> an existing course, 
revision of that course should be considered How- 
ever, the impact of the revision upon the original 
requirements for the course must be carefully 
studied. If the proposed changes do not detract 
from the original purpose, revision may be an 
effective method for meeting both the original and 
fcurrent requirements 

(3) New courses Often the only practical 
method for satisfying education or training re- 
quirements IS to develop a new course. This is 
largely due to the unique requirements imposed 
by the complexit> and sophistication of new s>s- 
tems. In establishing new courses, decisions must 
be made concerning the most effective means for 
meeting instructional requirements. These decisions 
involve ohoosing from among the six types of 
training described in chapter 3 of AFM 50-5/ 
USAF Formal School Catalog. Volume I, 1 Sep- 
tember 1974, including contract special traininJ^ 
(Type 1), ATC special resident training (Type 
2), resident regular training (Type 3), and field 
training (Type 4). 

d. Home Study iNSTRucnoNAi. Packages: 

(1) There are instructional situations where 
it is more practical to bring the instruction to the 
students than the students to the instruction. The 
home study approach is a proven idea, and recent 
developments in instructional technology have 
extended the possibilities of what can be included 
in such instructional packages. Slides with simple 
handviewers and audiotape cassettes make multi- 
media packages feasibly and usable almost any- 
where. Programmed instructional techniques can 
make the instruction student-centered and individ- 
ualized Simple responder devices can provide 



prompt feedback to the student as he progresses 
through the instruction. Through application uf 
the ISD process, instructional packages can be 
produced with confidence that the> will accom- 
plish what they are intended to 

(2) Home studx instructional packages are 
especially useful: 

(a) as short courses, for example, it be- 
comes very expensive for 'ihe amount of instruc- 
tion involved to bring students from all over the 
country to attend a one-day course; 

(b) as refresher courses where the train- 
ing is needed by man> who are wideh separated 
geographically, for example, an annual navigator 
refresher course; 

(c) as preparatory instruction for 'those 
who need to develop or improve prerequisite skills, 
for example, an English grammar review for those 
scheduled to attend a course in which the ability 
to write studies and reports is essential; and 

(d) for updating instruction to keep skilled 
persons abreast of new developments in their field, 
for example, instruction on new information about 
allergies and their treatment for practicing doctors 

e Job Peformance Aids. In some situations 
a job performance aid or set of aids may provide 
all the instruction necessary, and avoid the need 
for forrpal instruction Job performance aids can^ 
assume many forms. The'plate on a gas heater that 
provide^ instruction on how to relight the heater 
is a simple job performance aid. Figure 3-1 shows 
an example of a performance aid for a maintenance 
task. Figure 3-2 is a performance aid in decision 
logic form. 

3-4. Criteria and Constraints Affecting the Selec- 
tion of Methods. The selection of cost-efTective 
methods for acquiring qualified personnel is in- 
fluenced by a number of factors. Chief among these 
IS the job performance required. Several additional 
factors need to be considered. Some of these latter 
factors may impose constraints upon the selection 
of methods. The following paragraphs describe 
these considerations and show their interrelation- 
ship.- 

T*hc information presented is based on the premibc thai 
instruction is necessary to develop the qualifications 
needed. 
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Figure 3-h Example of a Job Performance Aid 



a. Trained Personnel Requirements 
(TPR). Thib consideration is concerned with the , 
number of quahfied personnel required. The TPR 
reflects time-phased requirements for the various 
AFSs needed to accomplish the overall Air Force 
mission. Based on TPR information, instructional 
system designers must evaluate the various alter- 
natives for acquiring qualified personnel. Among 



the alternatives to be considered are the relative 
suitability of. selective personnel assignment, OJT, 
the use of existing resident or field training courses, 
and the development of new resident or field train- 
ing courses. Generally, a requirement for large 
numbers of personnel to be qualified over a long 
period of time can best be satisfied by the use of 
resident courses. 
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Figure 3-2. A Job Performance Aid in ihe Form of a Decision Logic Chart 
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b. Target Populaiion This term refers to 
the prospective personnel input to the system, 
career area, or specialty In selecting methods, the 
suitability of existmii L0u^^c^ and the practicality 
of revising such courses, in lieu of developing new 
ones, must be assessed. One basis for making this 
a^essment is mformation obtained from aptitude 
^ tests such as the Airmen Qualifying Examination 
(AQE) or other tests developed by the Air Force 
Systems Command's Human Resources Labora- 
tpry, Personnel Research Division. Results of these 
tests are useful in determining the range of ap- 
titudes, prior experiences, knowledges, or skills 
possessed by the target population. This informa- 
tion has an impact on course len^zth and content, 
equipment and facility requirements and time 
needed for instruction. 

c Task Criticamty and LEARriiNG Diffi- 
culty: 

( 1 ) The criticality of a task is determined 
by two considerations whether the task is per- 
formed under emergency conditions, and the con- 
sequences of inadequate performance 

(2) Learning difficulty is generally related to 
the complexity of the task and to the types of 
knowledge and performance is shown in table 
5-1 This list is arranged in order of increasing 
difficulty. (This table Is discussed in AFP 50-58, 
Volume II.) Complex tasks usually involve ap- 
plicartion of several of these types of knowledge 
and performance. 

(3) Knowing the t^pcs of knowledge per- 
formance involved in perforniini^ a task helps the 
instructional system designer determine the best 
method of instruction. For example, resident 
courses are usually best suited for teaching prin- 
ciples (rule using) and developing motor skills 
(motor chaining) This is because such courses 
provide the equipment and other resources needed 
to teach principles and pr^xedures, and provide 
^^ufficient time for the development of skills. 

d. AvAiL\Bn.iTY OP Time This factor refers 
to the time available to develop tmd implement 
instruction A short leadtime may rule out some of 
the possible options in the design of instruction. 
It may also be appropriate to consider purchasing 
instruction (from the manuLicturer, a university, 
or .other appropriate agency ) oc hiring trained 
personnel to do the job until there is time to de- 
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velop an Air Porce-prepared program ur the in- 
structional rt^^quiremeiU ceases to exist (12). If 
instruction iS| to be purchased, whatever informa- 
tion hi\s beCii gathered relating to Steps 1 and 2 
of the ISD process will provide useful guidance 
in specifying what instruction to purchase. 

e. Availability of Qualified Instructors: 

( 1 ) This consideration is concerned with the 
number and qualifications of personnel needed to 
develop and conduct instruction If instructors are 
not available, a period of 9 to 12 months is gen- 
erally needed to obtain the personnel and qualify 
them as instructor^ If the prospective instructor 
must also become technically qualified in the sub- 
ject area or specialty to be taught, the time re- 
quired to obtain a proficient iiistructor is in- 
creased. In addition, consideration must also be 
given to the availability of personnel needed to 
accomplish detailed planning and development of 
the instructional system. 

(2) Should an instructional program require 
mcreased manpower authorizations for its opera- 
tion and .maintenance, these resources must be 
programmed and budgeted well in advance of the 
time additional instructors or ISD personnel are 
required. 

f. AvAiLABiLiiv oi' Equipment and Facili- 
riEs. Decisions relative to the selection of methods 
may be influenced by requirements for, and the 
availability of, equipment and facilities. Although 
a detailed instructional design has not been ac- 
complished, requirerp^nts fo^r facilities and major 
items of equipment can usually be identified The 
acquisition of these resources usually requires lead- 
time for development and installation. Therefore, 
consideration should be given to the practicability 
of modifying existing equipment and facilities. 

g AvAiLABiLiiY OF INFORMATION. The devel- 
opment or revision of ^^ourses may be affected by 
the availability of data. When related to time and 
cost, this consideration may become a determining 
factor in selecting methods. 

h. Programmed Student Load. This is the 
number of students that must be in a course at 
any one time to satisfy ^ the requirements for 
trained personnel. 'Student load has a direct bear- 
ing on ^the requirement for equipment, facilities, 
and ^qualifie^ instructor personnel For example, 
when considering the use of an exi,sting course, 
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thought must be given tu that course's capabiht) . 
tu handle an increased student load. If it is ajread> 
operating at maximum capacit>, an increased stu- 
dent load will create a need fc'r additional in- 
structors, equipment, or facilities * 

i. Costs. 

(1) This factor is presented last since virtu- 
ally all the other considerations have an impact 
upon it. pach of the possible methods for acquirmg 
qualified personnel must be evaluated in terms of 
its economy, effectiveness/ -and timeliness for 
meeting requirements The cost factor is par- 
ticularly important when considering the ad- 
vantages and/or disadvantages of bringing the 
student to the instructional activity as opposed to 
taking the instruction to the student. 

(2) As important a consideration as cost is, 
It must always be considered in context with qualit> 
of training. The goal must always be to develop 
and to provide quality training at least possible 
cost, 

it 

3-5. Preliminary identification of Resource Re- 
quirements. The selection of methods for ad- 
quiring qualified personnel, discussed previously, 
is influenced by a number of considerations and 
constraints. Among these are factors relating to 
requirements for equipment, facilities, and per- 
sonnel. The success of ari Instructional system often 
depends on the adequacy and timeliness of these 
resources. The total resources needed cannot be 
determined until activities required in Steps 3 and 
4 of the instructional s>stcm model have bcq^\ 
accomplished.'' However, leadtime requiremients 
may make it neccssarv io identify i-ome of these 
resources prior to finalizing these steps. 

a. Equipment. It is usually necessary to iden- 
tify the major items of system-peculiar or unique 
equipment early in the ISD process. .Most of this 
equipment will undergo a development cycle 
similar to that of a weapon system. That is, it 
has to be designed, produced, installed, and 
tested prior to course implementation. Failure to 
identify these items of equipment until course 
development nears completion will have a detri- 
mental effect on the instructional system's ability 

'Step 3, Develop Objectives and Tests: Step 4, Plao. De- 
velop, and Validate Instruction 



to provide qualified personnel in a timel> manner 
However, equipment should not be procured until 
the instructional objectives have been tentativel> 
developed to be sure that what is obtained sup- 
ports a valid instructional requirement 

b. Facility. Facility requirements, like those 
for major items of equipment, must be identified 
as early as possible. Early identification is nec- 
essary because of the time required to obtain 
funds and complete the construction or modifica- 
tion project. 

c. Personnel: 

( 1 ) Though the leadtime may not be as great 
as for facilities and equipment, early identification 
of personnel requirements is also .necessary. Of 
primary concern are the personnel needed to ac- 
complish instructional system design. Additionall>, 
in planning for initial instructor requirements, pro- 
vision should be made for sufficient time to Irain 
the instructors. This is particularly important if 
instruction is being developed for new specialties 
or systems and instructors must become technically 
qualified. 

(2) There may also be a need to identify 
support personnel such as typists, illustrators, 
photographers, and hardware fabricators. 

(3) The budgeting of additional or revised 
manpower authorizations must be accomplished 
well in advance of actual personnel requirements. 

d. Funding. The identification of requirements 
for any of the above resource > will usuall> have 
an impact on financial requirements. Budgets have 
to be prepared, or revised, and funds appropriated 
to allow for the procurement of equipment and 
the construction or modification of 'facilities. The 
establishment of personnel requirements will also 
affect funding requirements for payroll support. In 
addition, it may be necessary to provide funding 
for travel if the instruction is to be conducted by 
mobile training teams (12) or if students are to 
attend courses in a temporary duty (TDY) status. 
Since budgets are normally submitted and approved 
long before money is actually spent, the instruc- 
tional system designer must exercise great judg- 
ment in determining, as precisely as possible, what 
resources will be required. Careful planning will 
help to assure resources when they are needed — 
not too early or too late. 
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3-6. instructional Standards: 

a. Purpose and Scope of Instructional 
Standards: 

( 1 ) At this stage, the job performance re- 
quirements have been established and the inst^'uc- 
tional requirements determined. If the instructional 
requirements are to be met by formal course, CDC, 
and/or OJT, the instruction to be provided should 
be documented in an instructional standa^^l, using 
the data from the worksheet shown in^iigure 2-1, 
or equivalent. If an instructional standard already 
exists, its validity, or need for revision, should be 
determined by checking it against the mstructional 
requirements. 

(2) Examples of instructional standards cur- 
rently in use are the Spccialt) Training Standard 
(STS) and the Course Traimng Standard (CTS). 
The STS is^the controlling document required for 
airman basic resident, airman advanced resident, 
airman lateral resident, and CDCs. CTSs are used 
for airman supplementary, field, special 'training, 

*and"officer courses.^ 

(3) These instructional standards are spe- 
cialized publications used to standardize and 
control instruction. They arc also the means for 
coordination between those responsible for instruc- 
tion and the organizations for whom instruction is 
conducted Such coordination helps to ensure that 
the instruction is designed to meet the using or-^ 
ganization's needs Instructional standard^ list, by 
means of task and knowledge statements, the 
instruction required to qualify personnel forjife^ 
performance These listings constitute* the OJ^anti- 
tative requirements to be satisfied by thc^nstruc- 
tional system Instructional standards are also 
qualitative. They specify the degree or level of 
proficiency to be developed for each task or 
knowledge Some instructional standards also show 
the method(s) of instruction 1o be employed and 
the allocation of tasks and knowledges to formal 
courses or OJT. ' , * 



*The preparation and use of the STS IS prescribed by 
AFR 8-13, Air Force Specialty Tramtnf^ S tamhyds. E^ch 
major command that uses CTSs has its own prescriptive 
regulation. ^The principal difference, between CTSs and 
STSs IS that CTSs list tasks and knowledges attainable 
within the course, whereas 5JSs*list the qualitative and 
quantitative requiremcnts'^for each level of skilj within 
a specialty Chapters 8 and 9 of this manual contain 
Extracts from CTSs ^Chapter 9 contains an extract from 
an STS. and so does VoKime IT of AF^P 50-58. 
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b. PREPARAriON OF INSTRUCTIONAL STAND- 
ARDS: 

(1) It is possible to develop a CTS that^is 
quite descriptive of the planned instruction if 
behavioral statements are used instead of phrases 
augmented by a, (!ode key (see the example in 
chapter 8). The ST§ describes not a single course 
but a whole career ladder with several skill levels 
and possibly several courses. This basic ^com- 
plexity coupled with use of a code key^(*\v,hich 
inevkably is open to some degree of intercala- 
tion), and the fact that STSs serve multiple pur- 
poses, make the usual STS unsuitable for use in 
detailed planning of instruction. Such STSs only 
reflect m summary form the instructional intent. 

(2) The task$, knowledges, and attitudes to 
list in the instructional standard can be drawn 
frohi the instructional requirements that have been 
established. For the STS, the factors discussed 
earlier in this chapter in connection with the 
determination of instructional requirements will 
determine the proficiency codes shown in the 
different skill level and course columns of the 
instructional standard (see chapter 9). 

3-7. Summary: 

a. The definition of education or training re- 
quirements is the first major decision-making step 
ii) ISD These requirements are determined by 
analysis of the job performance requirements and 
information about the people who are to perform 
the jobs. A determmation must also be made 
concerning cost-effective nle^ods for acquirii\g 
qualified personnel A decision to employ aiormal 
course of instruction usually necessitates uie pre- 
liminary- identification of resource requirements. 
These resources include the major items of system- 
peculiar or unique equipment, the facilities to be 
constructed or modified, t}ie manpower authoriza- 
tions required ()6 include possible trade-offs from 
current programs) and -tlje personnel needed to 
develop and place the instructional system into 
operation. Since each of these resources represents 
costs, financial requirements must also be identi- 
fied and funds budgeted. 

b. Als.0 during this step of the fSDn^rocess, 
instructional standards are developed. These r 
standards specifv the behaviors (knowledges, skills, 
and attitudes) and the degree or level of pro- 
ficiency needed for successful job performance 
and course completion. ^ 
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CHAPTER 



Development of Objectives and Tests 



DEVELOP OBJECTIVES 
• AND TESTS 



4-1. Introduction. In chapters 2 and 3, the prod- 
ucts of Steps 1 and 2 were described. In Step 3, 
these products are used to develop: 

S 

a. objectives, through an analysis of .^ucation/ 
training requirements; ; 

b. criterion-referenced tests, which evaluate the 
student's attainment of the objectives^; 

c. diagnostic tests, which measutjb the student's 
mastery of the supporting knowledges and skills 
required to achieve the objectives; and 

d. survey tests, which determine t{ie knowledges 
and skills already possessed by the intended stu- 
dent population. 

4-2. Development of Objectives: ^ 

a. Objectives and Learning: 

(1) ^Learning a physical skill requires actual 
experience in performing that skill. Mental habits 
are also learned through practice. Even attitudes 
artf developed or modified as an individual reacts 
emotionally to a stimulus. It is implicit in these 
statements that for learning to take place, the 



learner must be actively involved in the learning 
situation.^ 

(2) Learning cannot be directly observed. 
But it can be inferred on the basis of observations 
of how the learner behaves after instruction in 
comparison with how he behaved before instruc- 
tion. The person who has learned something is 
somehow different from the person he was before 
learning took place. His behavior is different. 



Since instruction should shape the desired be-^ 



havior, the objectives of that instruction should 
(describe the desired behavior. Also, observable 
behavior is what must be used to judge whether 
or not the instructional objectives have been 
achieved. 

(3) An objective is a precise statement that 
answers the question: What must Ihe student do 
to show he has learned what he was expected to 
learn? It specifies a behavior to be exhibited. For 
the statement to be understood in the same way 
by all who work with it, it must also include the 



'See AF^ 50-62, Principles and Techniques of Inslruc 
lion, chapter 2, 1 April 1974 for a discussion of the 
psychology of learning. 
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minimum standard uf pcrturmantc ur pruficienc) 
expected, and the conditions under which the be- 
havior is to be exhibited. 

b. Hierarchy of Oiufxiivis Objectives can 
be specified at several levels. Objectives can be 
specified for: the basic skills and knowledges; the 
more complex skills which represent combinations 
of these basics; and the application of these skills 
and knowledges in actual job situations. Many 
terms have been used to describe the various levels 
of objectives (criterion, terminal, prirrmry, en- 
abling, supporting, secondary, etc.). In this man- 
ual, the terms criterion objective and siihohjective 
will be used onJy where it is necessary to show a 
hierarchical relationship between objectives (as in 
the example in figure 4-1). Whatever the de- 
scriptive term that might be used (criterion, 
terminal, primary, etc.), basically they are all ob- 
jectives and should describe behavior, conditions, 
and standards. ^ 

c Characterisucs of Objectives: 

( 1 ) Behavior performance. An instructor 
cannot read the mind of the student to verify the 
extent of the students understanding. It is only 
through some -overt (observable) activity on the 
part of the student that the extent of his knowledge 
Of skill can be measured. The statement that a 
student will "understand Ohm's law'' is an'example 
of an obscure objective. One instructor may feel 
uKit if the siudent recites the law it indicates un- 
derstanding. Another may say that the student 
should explain the law^ A third may contend that 
the only way the student can satisfy this objective 
is for him to use the formula to solve problems 
in electrical circuitry A more precise objective 
could read that the student will "use Ohm's lav^ to 
determine applied voltage when amperage and 
circuit resistance are known/' An objective, so 
stated, .lets the student, instructor, supervisor, 
measurement personnel, writers, and course man- 
agers know exactly what the student is to learn 
and what behavior the student should exhibit to 
demonstrate satisfactory achievement. Use, of 
action verbs reduces ambiguity and promotes the 
"tinderstanding of instructional intent. 

(2) Conditions. A properly prepared ob- 
jective also clearly states the limits and /or condi- 
tions within which the student will he expected to 
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perform. This portion of the objective describes 
the important aspects of the environment in which 
the behavior will be performed. What does the 
student have to work with*^ Must he select the cor- 
rect tools? Is he allowed to use notes he has taken 
on the subject? Are technical orders or checklists ' 
available to him for guidance? What information 
will be provided as a starting point? These are 
typical questions which must be considered before 
an objective can be put into final form. 

(3) Standard. A third requirement of a well- 
prepared objective is a clearly stated or implied 
standard of performance. This portion of the ob- 
jective describes how fast (in 1 hour, 60 words 
per minute, etc.), how accurately (8 out of 10, 
:i;.0OI, without error, etc.), or the actual re- 
sponses that will be considered acceptable. The 
standard is the criterion for determining whether 
the student has achieved a satisfactory level of 
performance. 

(4) Exarnple. A few examples of statements 
of objectives that include behavior, conditions, 
and standards are shown in table 4-1. 

(5) Wording. An objective which incorpo- 
rates all three characteristics will communicate the 
instructional intent to all who are concerned with 
the instructional system. In stating precisely how 
thtf student will demonstrate his understanding of 
a task, the system designer must not only have 
in mind what the required behaviors are, but also 
must transmit this information to others. The same 
is true concerning conditions. The designer must 
make clear the conditions impv^aed upon the stu- 
dent as he performs the end-product behaviors of 
an instructioucil requirement. Likewise, he must 
make certain that everyone concerned understands 
what degree of success is expected of the student — 
a standard of performance. The wording of the 
objective is most important since the statement of 
the objective affects other planning decisions, such 
as the amount of repetition or piactice to provide, 
the amount of instructional time, and the amount 
and kind of instructional aids required 

d. Uses of Objectives. When precisely stated, 
objectives serve a variety of purposes in the 
development of more effective instruction. Some 
of these purposes are to: 

( 1 ) establish a basis for instructional strategy 
decisions; 
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Table 4-1, Statements of Objectives 



BEHAVIOR/ PERFORMANCE 



Measure and record the thickness 
of 0 uniform steel rod. 

Devise on operations plan with 
appropriate missions for AF 
commands within the theater of 
operations 



Field strip and reassemble on 
M.16 rifle. 

State the total face value 



CONDITiONS 

using a standard micrometer 
calipers. 

when given a war-game problem 
with data on equipment, loca- 
tion, force disposition, etc. 



STANDARDS 



Measurement should be withm 
it.005" of actual thickness. 



Acceptable plan must account 
for. planned use of aircraft and 
support facilities; anticipated 
expenditures of supplies; coordi- 
nation of operations. Points will 
be awarded as indicated in the 
attached rating sheet. 



Identify when each of the following 
conditions4irises in the process of 
starting and of shutting down 
aircraft engines: 

• EGT rises too rapidly 

• EGT exceeds 780° C 

• Engine oil pressure remains 
below 5 psi 

• Loadmeter indicates zero at 
60% RPM 

• Fuel flow is excessive 

• EGT fails tu rise within 20 seconds! 
of placing starter switch in 
GND position 



- ... H 

under conditions of total ' 
darkness. 


without error, and within 
5 minutes. 


of any combination of 10 US 
coins that includes one or more 
pennies, nickels, dimes, quarters, 
half dollars and dollars. 


{Implied standard: without error) 


given a T-17 aircraft with oper- 
ating J91-T3 engines, and TO 
1T-17B-1CL-1. (checklist). 


Should be done without error 
and without assistance. Recog- 
nition and response must take 
place within the limitations pre- 
scribed in TO 1T-17B-1CL-1. 



(2) determine appropriate content of the 
in.struction; 

(3) establish a basis for instructional media 
decisions; and 

(4) establish a basis for criterion tests. 

In addition, objectives serve a valuable purpose 
when they are included in instructional materials 
that are provided to the student. Studies have indi- 
cated that communicatitig the intent of instruction 
to students enhances learning. 

e. Preparation and Documentation. 

( 1 ) The statement of an objective should be 
interpreted in the same way by student, instructor, 
and all others who must work with it. For this 
reason, vague words and phrases that can have 
several interpretations should be 'avoided. Ex- 



amples of these are statements which call for the 
student to: 

know 

have an tiwar^ness of 
have a knowledge of 
be familiar with 
understand 
^ develop an appreciation of 

make proper use of 

Instead, action verbs should be used because they 
are more specific. The verbs used should reflect 
actions that are observable and measurable. Each 
of the following verbs indicates a specific type of 
behavior which is observable and measurable 

adjust plan ^ 

compare predict 
design record 
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TASK DESCRIPTION WORKSHEET (EXAMPLEl 



The TOW 





statement 
of Cr(tenon 
Objectives 



BEHAVIOR /PERFORMANCE 


CONDI TION(S) 


> 

STANDARD(S) 









Figure 4-2. How fhe Three Ports o/ on OJb/ecfive ore Derived from the Task Description Worksheet 



file 
locate 

multiply 
name 



remove 

repair 

select 

solve 

translate 



(2) Objectives are developed through a care- 
ful analysis of the instructional requirements that 
were established in Step 2 of the ISD process. If 
the Task Description Worsheet (figure 2-1), or 
equivalent, was used to derive the instructional re- 



quirements, the same worksheet can be used as the 
source of the objectives. Figure 4-2 shows how the 
performance, conditions, and standards can be 
drawn from this worksheet. 

(3) Figures 4-3 and 4-4 show a worksheet 
that can be used to record an objective. A separate 
sheet should be used for each objective. (This will 
facilitate sequencing the objectives later during the 
instructional design.) The task/knowledge state- 
ments, as already noted, can be taken from the task 
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OBJECTIVE WORKSHEET 

INSTRUCTIONAL STANDARD NUMBER: CH52^3AIR75100.X 
TASK/KNOWLEDGE STATEMENT STATEMENT NO 

Select and use appropriate methods and 4f 
techniques to achieve lesson objectives. 



PERF0RMANCE/BEHAVIOR(S) 

Select an appropriate instruc- 
tional method to teach an objec- 
tive. 



CONDITiON(S) 

Three different assigned ob- 
jectives*^ 

For each objective, informa- 
tion describing student char- 
acteristics, avoilable re- 
sources^ and available Hme 
wHHje*"prDvrded: — - 

Accomplish without assistance 
Irpm the instructor. 



PROFICIENCY LEVEL 

3b 



STANDARD(S) 

For each of the three objectives, 
the method selected must be 
compatible with all of the fol- 
lowing: 

• the objective 

• student characteristics 

• available time 
'^^^•'avdMlable resources. 



OBJECTIVE #4f.l 

Given a lesson objective, supporting reference material. Information describing the student population, 
and a description of the time, training equipment and resources available, select an instructional method 
for a^fiievmg the objective that will be compatible with student capabilities and available time and 
resources. Do this for three different objectives, without assistance 

~ ^ - - „ _ _ _ _ _ , 

SUPPORTING SKILLS/KNOWLEDGES/ATTITUDES 

Identify characteristics of each of the following instructional methods, case-study^ demonstration- 
performance, field trip, guided discussion, laboratory, lecture, peer instruction, team teaching. 



figure 4-3. Use of^jOb/ecf/ve Workshoei (Example A) 



description worksheet, or equivalent When the 
instructional standard is prepared, the portion of 
that standard which the objective suppofts can be 
shown under the headings STATEMENT NUM- 
BER(S) and PROFICIENCY LEVEL(S). This 
provides a means for documenting how the instruc- 
tion supports the instructional standard. In this con- 
nection, it should be noted that there is not 
necessarily a One-to-one relationship between ob- 
jectives and task/knowledge statement in the in- 
structional requirements (or statement in the in- 
structional standard). Several objectives may be 
required to support a task/knowledge, or, con- 
versely, a single objective may support several 
tasks/knowledges. The worksheet (figures 4-3 
and 4-4) also allows for the listing of those sup- 
porting skills/knowledges/ attitudes which must be 
attained before the student can reach the ob- 
jective. The format in this worksheet js not in- 



tended to be fixed; variations may be appropriate 
in some situations, 

(4) For instruction involving the operation 
and maintenance of equipment, it is comparatively 
easy to develop bchaviorally stated objectives. For 
other kinds of instruction, it may be more difficult 
to do this, but the analysis of behaviors should be 
just as precise as for that involving equipment. 
Traditionally, education courses have had a fixed 
length, and often there has been no ciear-cut 
instructional criterion for successfully completing 
the course. In part, this is because such courses 
entail the learning of knowledges, skills, and at- 
titudes which are very general in application. Be- 
cause the instruction has gefieral application, it is 
difficult to establish realistic standards of pro- 
ficiency for the instructional objectives. Yet more 
and more educational institutions are finding ways 
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OBJECTIVE WORKSHEET 

INSTRUCTIONAL 'STAhioARD NUMBER: CH52-3AIR75100-X 
TASK/KNOWLEDGE STATEMENT STATEMENT NO 

Select and use appropriate methods and 4f 
techniques to achieve lesson objectives. 



PERFORMANCE/BEHAVIOR(S) 

Prepare a lesson plan. 



CONDITION(S) 

Use one of the objectives as- 
signed for objective #4f-l . 
Use instructional method se- 
lected to teach this objective. 

Collection of reference mate- 
rial, information describing 
student characteristics, avail- 
able time, and available re- 
sources. 

Accomplish without instruc- 
tor's assistance. 



PROFICIENCY LEVEL 

!3b 



STANDARD(S) 

Lesson plan will include: 

• instructor's preclass 
preparation 

• outline of instruction 
(sequenced in job perform- 
ance, psychological and/or 
logical order) 

• criterion test-item(5} 

• summary 

Plan will be compatible with the 
objective, instructional methods, 
student capabilities, and avail- 
able time and resources. 



OBJECTIVE #4f.3 

Using one of the objectives assigned in connection with Objective #4f-1, the instructional method 
selected for that objective, collection of relevant reference material, and information describing student 
characteristics, available time, and available resources, prepare a lesson plan without instructor's assist- 
ance. Lesson plan should plan for the Instruction necessary to achieve objective. Lesson plan will Include, 
instructor's preclass preparation, student motivation, outline of instruction, criterion test, summary. Lesson 
plan wUI be compatible vvlth the objective. Instructional method, student capabilities, an J available time 
and resources. 



SUPPORTING SKILLS/KNOWLEDGES/AHITUDES 

List purposes of lesson plan. 
List parts of a lesson plan. 

Accept need for lesson planning as Indicated by preparation of a lesson plan when the requirement Is 
optional. 

Apply learning theory in selecting means for. motivating students, sequencing instruction, ensuring 
active student participation and reinforcing Instruction. 



Figure 4-4. Use of Ofa/ecf/Ve Worksheet (Example B) 



to resolve these problems. Fixed length of instruc- 
tion is being replaced by fixed achievement re- 
quirements. Since the amount of time to attain 
fixed achievement is not uniform for all students, 
course length is now variable. In many cases, in- 
dividualized instruction has replaced group instruc- 
tion, thereby facilitating fixed achievement re- 
quirements and variable course length. A content 
analysis approach ^similar to that described m 
chapter 2) is used to establish a basis for the 
definition of objectives, so that criteria for suc- 
cessful completion of the course can be established. 



Even attitude and motivation objectives, which 
usually are considered the most difficult to pin 
down, are being specified, using a goal analysis 
approach (described in chapter 2). 

(5) Analysis of the job training require- 
ments for many jobs will reveal eleyuents that are 
common to a number of tasks As examples: an 
electrician must be able to read wiring diagrams 
in connection with many of the tasks he performs; 
a number of the instructor's tasks require that he 
appl> motivational techniques, and reading and 
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interpreting the aitimetcr is common to man) of 
the pilot's tasks. It may increase the efficiency of 
instructional development to identify these common 
elements and specify objectives for them 

4-3. Identification of Supporting Knowledges and 
Skills. Once objectives have been developed and 
documented, the knowledges and skills which sup- 
port each objective can be; identified Each ob- 
jective is examined to determme intermediate 
steps (subobjectives) in the learning process which 
must be attained before the student can success- 
fully accomplish the objective. Some of these sub- 
objectives involve intermediate skills which mubt 
be mastered before an objective can be accom- 
plished. Others pertain to specific concepts, prin- 
ciples, and facts which must be learned Thus, the 
subobjectives pertain to;^th skills and knowledges, 
which are identified inrough a behavior analysis. 

a. In its purest sense a behavior analysis is an 
analytical process based on empirical data. Its 
purpose is to determine the skills and knowledges 
directly related to task performance. The final 
product cif a behavior analysis will include three 
things: 

(1) A complete, precise listing of each skill 
and knowledge- that must be taught in order to 
prepare the student to perform the task. 

(2) A determination of the environment or 
conditions under which the student will perform 
the task. 

(3) The factors which constitute acceptable 
performance by the student. 

The analyltical process consists of a review of all 
course control documents, field surveys, tasks 
observation, and any other process which will 
lead to empirical data about the actual perform- 
ance of the task. 

b. Identifying the skills and knowledges neces- 
sary for attamment of an objective is rarely a 
simple process, Becau'^c skills can be observed and 
knowledges cannot (and therefore must be in- 
ferred). It IS more difficult to identify precisely 
what knowledges are needed to satisfactorily per- 
form a job task than to identify the skills required. 
The only knowledges that should be mcluded in 
an instructional program arc those required in the 



performance of a job task or tasks (manipulative 
or mental). 

c. The following list of questions may be used 
as a means for identifying job and task behaviors for 
which knowledge may be reqtlired (13). These 
questions do not provide a method for determining 
what specific knowledges are required to per- 
form a task. Rather, they are mtended to help 
identify those job activities for which knowledge, 
of a kind pertinent to a particular question, will 
likely be of assistance in learning and performing 
the activity. 

(1) is a knowledge of nomenclature and 
location of components required? 



(2 ) Are safety precautions involved? 

(3) is the interpretation of symbols or 
Mgnals necessary? 

(4) Do calculations have to be performed? 

(5) Are the techniques of problem solving, 
diagnosing, and troubleshooting required? 

(6) Does the task require the anticipation 
of later conditions from earlier conditions? 

(7) is planning involved in the accom- 
plishment of the task? 

(8) is the selection of strategies involved? 

(9) Must the task be performed in accord- 
ance with briefing instructions? 

(10) Is the use of tools and general test 
equipment required? 

(11) Is knowledge of the why of a' procedure 
essential? 

In answering this last question, caution should be 
exercised to avoid including unnecessary instruc- 
tional content. On the other hand, it would seldom 
be economically practical or even educationally 
sound to teach a student a complex task by merely 
having him duplicate steps of the task as they are 
performed by a technician. He should be taught 
the basic skills and knowledges required on the 
job, that is, to perform simple motor acts, follow 
technical order procedures, identify components, 
and state a fact or explain a principle. 

d. An examination of the objective shown in 
figure 4-4 suggests that supporting skills, know!- 
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edges, and attitude:^ must be acquired before the 
student can achieve the objective These are iden- 
tified in the bottom block of figure 4-4. When 
stated in behavioral terms, thev represent objec- 
tives of instruction which must be attained as in- 
termediate steps in the learning process Finally, 
statements resulting from the analysis of these sub- 
objectives should identify the basic skills, knowl- 
edges, and attitudes needed These should include 
the kinds of discrimnnations which must be made, 
the decisions necessary, the identifications needed, 
the prmciples and facts to be learned, the pro- 
cedures to be followed, and the motor responses 
required. 

e Dnee the objectives have been analyzed, tests 
can be developed to measure the student's achieve- 
ment of these objectives. 

4-4 Development of Tests. Tests serve a variety 
of purposes. This portion of the manual will ex- 
plain the development of criterion tests, diagnostic 
tests, and survey tests'. , 

a Criterion Tlsts CYiterion test items are pre- 
pared and administered to evaluate the student's 
attainment of the objectives and to measure the 
effectiveness of the instructional system. 

(1 ) Characteristics. Each test item should be 
based solely on the requirements specified in the 
objective which the item is to measure. Thus, 
an appropriate test item for the objective "Parallel 
park a car between two cars without hitting either 
car" would be to require the student to actually 
parallel park 'a car between two cars. Multiple 
choice questions about parallel parking would not 
be an appropriate measure of the student's attain- 
ment of the objective. Neither would requiring the 
student to parallel park a car that has a trailer 
attached. 

(a) Once it is determined that all objec- 
tives are essential, a test should be prepared which 
will measure each objective within the parameters 
established by the objective. To show that he has 
atttained the objective, the student must either 
meet or exceed the specified standard of per- 
formance. This is basically a go, no-go type of 
measurement. How well one student does is not 
determined by how the other students- do. Tests 
which rate individuals with respect to standards of 



performance that are specified in objectives are 
called criterion-referenced tests. 

(b) Norm-referenced tests-^tests which 
are '^graded on a curve" — serve a very different 
function from criterion-referenced tests. Norm- 
referenced tests compare how well one student 
performs with the performance of others in his 
group. Although norm-referenced tests are use- 
ful in providing a relative standing among stu- 
dents, they have little value as a quality control 
for the evaluation of an individual or an instruc- 
tional system. Criterion-referenced tests are the 
only ones appropriate for measuring attainment of 
objectives and effectiveness of instruction. For 
those situations where relative standing among stu- 
dents may be needed, norm-referenced testing may 
be used as a supplement. 

(2) Preparation. To emphasize a point made 
earlier, each criterion test item should be based 
solelj^ on the requirements specified in the ob- 
jective which it is to measure. In fact, the word- 
ing in both the test item and the objective may be 
identical. In an automotive course, for example, 
one may find the following criterion objective. 
"Using an electrical test set, adjust the voltage 
regulator until the voltmeter indicates 14 volt^, 
within, a tolerance of ±.2 volt." In this case, 
the objective itself is a good test item; in others, 
some rephrasing may be necessary. In any case, 
a test item should require students to respond only 
as specified, under established conditions, and with 
the minimum level of acceptable performance.- In 
the objective cited above, the student is expected to 
perform the task. The corresponding test item 
should require identical performance^ by the stu- 
dent — not merely have him state the procedures 
that are involved in the adjustment of the voltage 
regulator. 

(a) For some objectives involving safety 
or very expensive equipment, it is necessary to use 
simulation in the instruction, and it is also neces- 
sary to use simulation in the testing. Thus, the 

^Minimum level of acceptable performance does not 
imply weak or inadequate performance. The minimum 
acceptable performance may be perfection. 
^Performance is not limited to physical manipulations 
such as writing or weldinjg. There also is performance 
when one adds numbers, makes a judgment, etc. Per- 
formance of the latter kind^can only be observed m 
directly as when the sum of the numbers, or the judgment, 
is written or spoken. 
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student's abilit> tu defuse a bumb prubabl) \vould 
be tested using a simulator. Another t>pe of situa- 
tion is found in some education courses that deal 
with job-world problems Here it may be neces- 
sary to test with a simulated problem, such as a 
case study, instead of using the actual situation. 
For exam{51e, a squadron officer course might in- 
clude the management decision process involved 
in setting priorities. It may not be possible to let 
each student be a ''squadron commander for a da>" 
and actually face the real problems. In this case 
a simulated performance test could be created; 
the student is seated at a ^'squadron commander's 
desk" with an inhasket of simulated correspond- 
ence requiring action, and some fellow-students 
play the roles of squadron members who have 
problems to take up with the "commander." In 
each of the above examples the behavior in the 
simulated situation, and the standards of perform- 
ance, are the same as in the job setting; but, the 
conditions under which the behavior is to be ex- 
hibited are only similar. It must be kept in mind, 
however, that the objective defines student per- 
formance in the instructional (as opposed to job) 
setting. Objectives specify what is to be achieved 
during instruction that will enable the student to 
perform adequately when he is on the job. 

(b) Other alternatives would be to measure 
the subordinate competencies of the objective sep- 
arately or to measure some performance which is 
known to be correlated with the criterion perform- 
ance of the objective (14). An example of the 
latter would be where the job performance require- 
ment is the ability to troubleshoot a piece of com- 
plex equipment such as a radar set. It is not 
practical to test the student on his ability to suc- 
cessfully troubleshoot every possible malfunction. 
Instead, the student might be required to trouble- 
shoot a limited number of malfunctions which are 
considered typical in terms of the basic skills and 
knowledges involved 

(3) Contitrrente, When the objectives have 
been established and criterion test item^^ devised 
to measure them, the objectives and test items 
must be approved b> the authority designated in 
related directives.^ With this approval, the ap- 
proved objectives/criterion-test-items package be- 



*Concurrcnce of the using organizations is obtained 
through coordination on the instructional standard. 



comes the agreed target, instruction must then be 
designed to enable the students to reach that target. 

b. Diagnostic Tests. 

{ 1 ) As indicated earlier, the criterion test is 
used to help the instructional system designer eval- 
uate the student's attainment of the objectives and 
to validate the instructional system. T^t items must 
also be prepared to determine attainment of the 
supporting skills and knowledges which contribute 
to the ability to perform the objectives. As with the 
criterion test items, these diagnostic test items 
should measure precisely what is to be taught — 
no more, no less. 

(2) ^During validation, the instructional sys- 
tem designer can effectively use diagnostic test 
items to predicjt success, to identify problem areas, 
and to adjust instruction for unanticipated indi- 
yidual differences Once the instruction is vali- 
dated. It would seem that there would be no further 
need for the diagnostic tests. For if the student 
can pass a criterion test item, this should be evi- 
dence that he not only has attained the related 
objective, but also that he has mastered the com- 
ponent skills and knowledges. However, if the in- 
structional testing of a validated program is limited 
to criterion test items, there can be a delay before 
it is discovered that a student has not learned. 
Judicious use of diagnostic tests is appropriate, 
and will avoid this problem. But extensive use of 
diagnostic tests during instruction, when the in- 
struction has been validated, is costly and un- 
necessary. 

c. Survey Tesps. 

( 1 ) Purpose Survey tests are designed to 
determine what prospective students (the target 
population) already know or can do befoj;eTcceiv- 
ing instruction They should not be confused with 
pretests, though the design of both may be similar. 
The survey test is administered while the instruc- 
tional system is being developed and provides valu- 
able clue^'tor-hsyiesign. The results of the survey 
will aid in dedKing which objectives require in- 
struction and^hich can be deleted because the 
student population has already mastered them. 

(2) Preparation. The survey test should in- 
clude items selected from the criterion objective 
examinations. Idctllly, the ability to accomplish 
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all cnterion objectives should be measured In ad- 
dition, the test should survey related abilities of 
the prospective student, such as reading, writing, 
and jnaking certain mathematical calculations if 
these abilities are required in the attainment oi 
course objecti\es. The degree to which .survey test- 
ing can be effectively accomplished, however, is 
contingent on several factors, such as the avail- 
ability of: 

(a) prospective students 

(b) valid tests (in-house or commercial) 

(c) facilities 

(d) equipment (when testing perform- 
ance), e 

(e) funds 

(f) time. 

(3) Use of test results As indicated before, 
survey tests can be used effectively to determine 
course content and depth of instruction, especially 
In areas af basic skills and knowledges. For ex- 
ample, if It IS determined that nearly all of a 
sample of prospective students can accomplish a 
particular task or that they know a certain piece 
of information, then this skill or knowledge may 
not have to be taught. Conversely, if few can 
exhibit mastery of a particular skill or knowledge, 
then, more than likely, the corresponding objec- 
tives should be included in the instruction. How- 
ever, decisions on what instruction should be de- 
leted or included should not be made arbitrarily 
even when nearly all of the students sampled can 
satisfactorily perform a certam task, without con- 
sidering such factors as: 

(a) How can instruction be econpmicully 
provided to the few who do need it'^ 

(h) Are there seasonal varmtions in the 
aptitudes of the prospective students'^ 
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(c) How difficult IS the task to learn? 

(d) How critical is the task to mission 
success? 

These and other questions must be answered be* 
fore system designers can make valid decisions to 
delete objectives from an instructional program. 

d. Pretests. Pretests are designed using pro- 
cedures similar to those used for survey tests Stu- 
dent ability to achreve each criterion objective 
should be measured. Generally, a pretest is used 
after the instructional system is in operation. When 
planned pacing is used to accommodate the vary- 
ing needs of students, the pretest is administered 
to each student prior to instrbction to determine 
the extent of his knowledge and skill relevant to 
that instruction. How well the student performs on 
the pretest determines what instruction is then pro- 
vided. •* 

4-5. Summary. Obj'ectives represent the specific 
intent of instruction in terms of behavioral end- 
product based on job requirements. Subobjec- 
tives establish the intermediate skills and knowl- 
edges necessary to satisfy the requirements of the 
criterion objectives \n and of themselves, objec- 
tives have little significance if tests are not de- 
veloped which will precisely measure the behaviors 
specified by the objectives, as well as validate the 
success of the instructional system These are the 
functions of criterion tests. When practical, survey 
tests are prepared to determine which objectives 
should be retained in the instructional system. Pre- 
tests may also be prepared at this time if it is an- 
ticipated that an individualized instructional de- 
sign will be incorporated in the next step. Once 
these activities have been accomplished, the system 
designer can plan, develop, and validate in- 
struction. 



,er!c' 



38 



AFM 50-2 



5-1 



CHAPTER 



Planning^ Developing, and Validating 

Instruction 



^ ?f|rAWNGaEQU|«IMEi!»ISt 



CEVEWPOWECTIVES 




PLAN. DEVELOP, AND 
VALIDATE INSTRUCTION 



5-1. Introduction. The results of Step 3 arc a set 
of objectives with related test items These ob- 
jectives and tests iucntify what is to be taught 
The contents of this chapter will be concerned with 
how the material will be taught, and how to de- 
termine whether or not the instruction is effective 
The acHvities discussed in this chapter include 

a. how to plan and develop the most efficient 
and effective learning environment utilizing prin- 
ciples of learning; 

b. how to sequence instruction to provide for 
optimum learning; 

c. how to select instructional methodology, 
equipment, and aids" -^o facilitate the learning 
process; 

d. how to determine resource requirements; and 

e. how to validate the instructional svstem 



5-2. Planning Instructions: 

a. Managemeni oi- Llarning. Learning is 
defined as a change m the behavior of the learner 
as a result of his experience. In accomplishing 
Steps I and 2 of the ISD process, both the desired 
behavior (job j^f^rfnrniance requirements) and the 
current capabilities of the target population were 
identified. The difference between the two is the 
learning deficiency that must be made up by in- 
struction. This instructional requirement was then 
described as a set of objectives (Step 3). Now 
t-hat the instructional designer knows what be- 
haviors must be shaped, he must plan the means 
for doing this. The instructional planning should 
use the "following concepts for cfTectively manag- 
ing the learning process. 

( 1 ) Active student participation. The in- 
structional designer and the instructor can only 
arrange for learning to ta)ve place It is the student 
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who does the learning. He learns by doing, think- 
ing and feeling—]^ an,s ^ criag.,qucstiunv, Ji^cu^- 

ing, computing, manipulating, putting ideas to- 
gether, etc.— provided that the spp^ific actnitv is 
appropriate to what he is trying^u loarn Until 
the student has applied in an appropriate \va> what 
he was (aught, it is a risk) presumption to assume 
that he has learned 1, earning is an indi\idual 
process in which students acquire knowledge.s. 
skills, and attitudes through ^^eir own activities, ex- 
periences, and motivation 

(a) When the teaching ot knowledge is 
critical,' learning can he enhanced by using ques- 
tions throughout the lesson. Each question should 
"^pport one or more learning objective and require 
an answer that is within the scope of the teaching- 
learning activity. Learning can be further increased 
by requiring each student to lespond 

(b) When a skill js being taught (for ex- 
ample, use of water fog to extinguish a class A 
fire), each student should accomplish every step 
of the exercise; performance is essential In addi- 
tion, techniques such as demonstration and films 
can often be used to assist the students in formin]g 
a mental picture or in remembering a process In 
situations where complex tasks are taught, an in- 
structor may Keed to lead stude/its through each 
step of the -exercise before allowing them to pro- 
ceed on their own. 

(e) Because definition and measurement 
of attitudes is more diflicult, designing instruction 
to shape attitudes is also more difficult However, 
as with >kiiis and knowledges, if desired attitudes 
are to he developed and or undesvrahle attitudes 
extinguished, it is equall) necessar> to pldu suit- 
able instructional experiences One effective ap- 
proach to shaping attitudes js human modeling — 
setting or otherwise providing [\ desired example 
for the student to follow Guided discussion also 
works effectively for affective lessons, as it requires 
the student to support and defend their respective 
positions on the issue of concern YcX another 
proven approach is the appropriate application or 
witfiholding of rewards (Ch^ipter 7 includes sug- 
gestions for designing instruction to shape atti- 
t ij/des. ) 

(2) Confinnanon (knowledge of result's). As 
the student participates, through various types of 
responding, it is helpful to him/that is, contributes 
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to more efficient learning) to know how well he is 
-jluiug — CaalkmaUun-Coiisiiktixujf-anfuriuiug-lhe^tu^- 
dent whether or not his response is correct. This 
kind of feedback ma> also provi^le a valuable 
source of motivation. The research is not as clear 
when it comes to the specifics of how to use eon- 
tirniation in instruction. The general conclusion 
seems to be that in the beginning stages of instruc- 
tion, when it is important to build confidence in the 
student, confirmation should be provided for oaeh 
response, As the instruction progres^se^ eon forma- 
tion may be provided on an intermittent schedule.' 
In considering if confirmation should be provided 
immediately after the student has made his re- 
sponse, the same rationale would apply. The type 
of learning involved, the distribution of practice, 
and the complexity and method of presenting the 
subject matter will necessitate variations in the time 
interval between student response and confirma- 
tion. Knowledge of results may be provided in sev- 
eral forms. For example, it may arise naturally out 
of a task environment where a student is required 
to select the switch position that turns on a specific 
light. When the light comes on, the student knows 
he has made the right choice. During a later ex- 
ercise when the student is wiring an electronic cir- 
cuit, confirmation can be supplied by the instruc- 
tor He ma> merely check the individual's work at 
appropriate intervals and let him know if he is 
proceeding in the correct manner. When discover- 
ing an error, the instructor should point out the 
mistake and provide sufficient individual tutoring 
to allow the student to continue the exercise, fn 
another situation where students are using a re- 
sponse device to answer questions^^ during a dis- 
cussion or demonstration, knowledge of results 
may come from the device itself or it ma\ be sup- 
plied by the instructor. 

(3) Planned pacing. The rate at which stu- 
dents proceed through an instructional sequence is 
referred to ;k pacing. There are several ways to 
pace students' progress. 

(a) Traditionall>, instruction has been 
presented in a lockstep fashion; that is, all stu- 

^Remforcement is often associated with confirmation, but 
tlie two terms aie not synonymous Reinforcement has a 
specific effect on behavior. It makes that behavior more 
hkelv to happen again when the same stimulus is present 
Confirmation may be a means for providing rcmforce- 
ment, but it is -not the only means. 
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dents in the clai>s ceccived the scuiie instruction 
-at the-samc time However, this method of con- 
ducting instruction is inappropriate in man} teach- 
ing-learning i>ituations Wticn all the studenti> in a 
class are required to progress at the same rate, 
frequently some will become disanfraged, and their 
performance will probablv be iHinmial The slow 
student will become bewildeptd and the student 
with the higher aptitude, disinterested '^herc are, 
however, certain kinds of learning experiences 
which the students must share In situations where 
there must be an exchange of de is. discussion, or 
debate^ the instructional environment require^ a 
jrptip. Likewise, many ta>ks impo>e physical limi- 
tations that are bevond the capabilities of one or 
two people^ Therefore, to meet job pertormance 
requirements, instruction on such tasks should be 
accomplished in/Tgroup 'netting 

(b) All students cannot achieve the re- 
quired level of performance of a task in equal time 
Because of varying ability, background ;ind moti- 
vation, some will learn faster than others Ideally, 
then, the mstrucjional designer would plan for each 
student to go through the instruction at a pace best 
suited to the student Although self-paced instruc- 
tion is desirable, application of this pacing method 
to complete courses is pot alwavs practical First, 
the teaching-learning situ'ition may recj^uire a group. 
Second, in a completel) self-paced course, stu- 
' dents will complete program m various lengths 
of time This can create student graduate assign- 
ment problems in the case o\ pipeline stvidents 
This constraint can he overcome by realigning as- 
signment policy to synchronize wyth production 
rate. Other limiting factors m the use of self- 
paced instruction may be the availability of equip- 
ment, manpowci, facilities, time, and funds, Lack 
of a feeling of group identific ition may also be a 
problem in some cases Where it*is possible to 
use sclf-pacecl instruction, usual I v there is a re- 
sultant saving in training time However, when 
onl) the beginning of a cour'^c is self-paced,faster 
students must often wait for the slower ones before 
entering the lockstep portion, and that does not 
,pr^idc a cost-elTective system. 

(cK^ince the lockstep approach hampers 
the slow-as vvt-ll as the fast students, and the self- 
paced approach ma> be limited b) certain con- 
straining factors, a commonlv adopted approach 



IS to use a compromise between the two methods. 
Fortunatelv, manv inbtnictional s>stem.s can use 
planned pacing which combines the lockstep ap- 
proach where necessat-v and seil-paced instruction 
wherever possible, but the cautions njentioned 
above must be kept in mind 

(d) Several variants ot planned pacing can 
be used to accommodate the vpryirtg needs of 
students. 

Proficiency Advancement. This is an 
acceleration technique which <fan be used to ad- 
vance students who have prior knowledge, prac- 
tical experience. Or who are exceptionally rapid 
learners. Students -demonstrate their proficiency 
bv accomplishing the criterion test for a particular 
instructional sequence. The students are advanced 
accordingly through each sequence i^ which pro- 
ficiency has been satisfactorily accomplished One 
variation of this technique is where the student is 
allowed to complete one or more mstructional se- 
quences of a course on his off-duty or remedial 
study time by studying the regular course niateriajs. 
Again, the student is moved ahead through each 
sequence in which he demonstrated satisfactory 
achievement Proficiency advancement is an espe- 
cially plausible function for u^e in upgrade train- 
ing programs. 

_2 Multiple Tracks: 

The technique of using multiple 
tracks allows students with varying capabilities, 
prior knowledges, and learning characteristics to be 
placed in an instructional track suited to their 
needs and appropriate to their abilities. Practical 
considerations generally necessitate limiting the 
number of tracks to two. three, or four Usually, 
student placemeat in these tracks is determined 
through the use pf a pretext as discussed In chap- 
tcr 4. ) ; 

Multiple tracks could also be used 
where it is known that foiiowmg graduation the 
students will have different assignments. For ex- 
ample, weapons mechanics (AFSC 46230) who 
afe to be assigned to Air Defense Command, Stra- 
tegic Air Command, and Tactical Air Command 
following completion of the basic course have dif- 
ferent training needs. ^>After receiving instruction 
based on common needs, each group is placed on 
a different track to receive the instruction that is 
command-peculiar 
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J? Modular Scheduling: 

a_ In modular spfieduling, the instruc- 
tional system is divided mtu units of instruction 
called modules. Each module covers one or mure 
objectives. Students are pretested and counseled 
to determine their particular instrutdtional needs 
Then, an individual program is designed for each 
student. Pretest results and information obtained 
through counseling are analyzed to determine the 
specific modules needed b> each student. Struc- 
turing a jourse in this manner will eliminate non- 
essential instruction and can reduce average course 
length. 

^ In equipment-oriented courses in 
which the sequence of units studied is mi critical, 
modular scheduling can assure fullest utilization of 
the training equipment' Put another way, modular 
scheduling may make it possible to manage with 
less training eqiiipment.lj^ 

4_ Self-Paced Instruction. Programmed 
texts and other programmed instructional materials 
can be advantageously used (b enable each student 
to progress through the instruction at his own pace. 
They can be designed to use or not^to use the 
assistance of an instructor. Materials that do not 
require any instructor assistance can 1)e used for 
out-of-class study. Normally, there still is a need 
to set some limits on how much time the student 
may take for the instruction. If this is not done, 
and students are given complete freedom to set 
their own pace, the less-motivated may take an 
inordinately long time to complete the instruction. 

_5 Learner-Controlled Instruction. 
Where the students are highly motivated and one 
of the desired outcomes of instruction is that the.y 
develop self-reliance and initiative, learner-con- 
trolled instruction has been used successfully In 
learner-controJled instruction, the student is given 
the set of behaviorally stated objectives *and a 
lest item for- each so that he*^ can determine when 
he has attained the objectives. He is also informed 
as to what learning resources are available and 
where they are. He is free to use any part, all, or 
none of the resource materiaij? as he deems Neces- 
sary. When he feels ready, he goes to the instruc- 
tor to take the unit test. 

b. INSTRUCTIONAI INCREMENTS— SeQUCNCH 

AND Size- - 

( 1 ) Optimum instructional sequence. 



( a ) Since the objectives provide the frame- 
work for the structure of the s>stem, the sequenc- 
ing of objectives become^ a \er> important activity 
in system development. The sequence of instruc- 
tion can greatly affect the motivation of a student. 
Sequencing can emphasize relationships to make 
instruction more meaningul Obviously, the more 
meaningful the content, the easier it is to learn. 
Proper sequencing also avoids unnecessary dupli- 
cation in course content. Careful sequencing also 
prevents gaps in instruction and makes certain 
that the development of a skill is orderly and that 
prerequisite knowledges and skills have been ac- 
quired prior to the introduction of advanced sub- 
ject matter content or the performance of more 
complex tasks. 

(b) Course content can be sequenced in 
any one or a combination of several ways. 

W Job Performance Order. One method 
of arranging instructional content is in the sequence 
in which tasks and task i?Iements are performed. 
This places instruction, insofar as possible, within 
ihe context of the work environment. As the stu- 
dent learns new tasks, the tasks are related to 
the duty, while the task elements (knowledge and 
skill components) are related to the task being 
learned (15). Learning tasks agd task elements 
arranged in the same sequiencq in which they are 
performed in the work environm^t lends a great 
deal of realism to instruction. It also provides for 
better transfer of learning from the instructional 
to the duty environment. This method of sequenc- 
ing would be especially applicable to teaching pro- 
cedures composed of a series of fixed steps. The 
knowledge and skill elements of one ta3k can 
quickly be transferred to another related task. 

J_ Psychological Order. This method 
of arranging instructional content is based on ease 
of learning. This means that, when possible, the 
learner moves from the simple to the complex, 
fr®m the known to the unknown, or from the 
concrete to the abstract. At the start of any instruc- 
tional J^equence, motivation can be enhanced if 
the student can relate prior knowledge to that being 
taught. Likewise, instruction on complex motor 
skills, generally, is more effective when begun with 
transferable simple motor activities. For example, 
ii student should be taught to use certain tools 
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on simple tasks betore attempting to repair a com- 
plicated piece of equipment. 

^ Logical Order. Often mstructjonal 
content falls mto a logical pattern because oi the 
combined elements of job performance and psycho- 
logical order. For greater learning effectiveness, 
instructional activities should normally proceed 
from^he simple to the complex Unfortunately, 
many learning activities do not lend themselves 
to such an arrangement. Instead of gradually m- 
creasing in difficulty as shown on hne 1 of figure 
5-1, job ordered tasks often create a difficulty pat- 
tern similar to that shown one hne 2. When de- 
veloping instruction for such an exercise, the sys- 
tem designer may wanf to keep the job tasks in 
their proper sequence, but still allow the students 
to proceed from the simple to the complex. This 
can be done by an arrangement of instructor-stu- 
dent activity where the instructor first accomplishes 
the more difficult parts as the student observes. 
If it is necessary for students to cofnplete the entire 
procedure, several operations may have to be de- 
signed with a planned decrease in instructor par- 
ticipation for each operation as illustrated in 
figure 5-1. 

This incorporates the whole-part-whole concept. 
For example, when teaching the assembly of a 
carburetor, it probably would be best to begin by 
demonstrating the entire assembly procedure; then 
to break it into a step-by-step demonstAtion inter- 
spersed with step-by-step participation; and finally 
to recombine all the elements in a complete run- 
through of the assembly procedure. However, 
variations of the whole-part-whole concept may 
have to he applied in certain situations. As men- 
tioned earlier, if a task is too long to be learned 
effectively as a unit, it may be desirable to divide^ 
the task into several operations — always ^remem- 
bering to emphasize the relationships to the whole 
task. Finally, the entire task can be practiced in 
its entirety. 

(c) In actual practice, all types of se- 
quencing have a place in course development. The 
type of sequencing used should depend upon the 
nature of the task or knowledge being taught and' 
\ the availability of resources. The following gen- 
eral rules should be coasidered during the prooess 
of sequencing: , ^ 

ERLC 



J_ Place easily learned objectives early 
m the sequence. 

2, Introduce concepts at the first point 
where the understanding of the concepts is a pre- 
requisite for successful performance. 

J_ Introduce instruction on prerequisite 
skills prior to the time where they must be com- 
bined with other skills and applied. 

4^ Place procedural skills and knowl- • 
edges (within the limitations of equipment and 
facilities) in the same sequence as required in the 
work environment. ; 

5 Introduce a knowledge or skill in the 
task in which it is most likely or most frequently 
to be used. ' \ 

^ Provide for practice of skills and con- 
cepts in areas where transfer of identical or re- 
lated skills is not likely to occur. 

7^ Place complex and cumiilative skills 
late in the sequence. 

(d) The documented objectives should 
now be arranged in an order consistent with the 
sequencing rules just discussed. \ 

J_ First, arrange related objectives- in 
homogeneous groups. For example, the grouping 
may be by subject, Jask, 6r equipment subsystem , 
such a ^an a^ment of human resourcesj^^^ecurity, 
radar^WMi||tion, supervision^ landing gear, and ^ 
inertial guiaance. 

_2 Next, the instructional units should 
be placed into a sequence which will provide for 
continuity of instruction, ease of learning, and dis- 
tribution of practice. However, time limitations 
and availability of resources must be considered 
If there are skills and knowledges that are com- 
mon to a number of tasks, generally these should 
be pulled out and taught before the first task in 
which the common element is required. (See AFP 
50-58 for guidance on how to identify and handle 
common-element instructional objectives.) & 

^ Last, the objectives should be se- 
quenced withm each unit of instruction, stijl tak- 
ing into consideration ease of learning and con- 
tmuity of instruction. Again, time and availability 
of resources may be limiting factors. The sequen- 
tial arrangement of objectives within each unit of 
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LEARNER DOES ALL OPERATIONS - INSTRUCTOR CHECKS CLOSELY 



Figure 5-]. How to Teach DtfTicuh Operations 



instruction will provide the input to later develop- 
ment of a plan of instruction or syllabus. 

(2) Optimum-size instructional increments. 
Another important concern in ISD is the proper 
size of each increment of instruction. The term 
optimum IS used to indicate that amount of in- 
struction which is necessary to produce desire4^ 
results. The natural tendency is to "play it 'safe'' 



and provide more instruction than is necessary. 
This makes training more costly than it need be, 
and may cause boredom and frustration, espe- 
cially among the more able students. The best 
approach is to design minimal instruction initially, 
and rely on the validation of the instructional ele- 
ments (a process discussed later in this chapter) 
to reveal where additional instruction is needed. 
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This process plus additional tcedback. from evalua- 
tion of instruction, should determine the optimum 
amount of instruction. If more instruction than is 
necessary is' provided initidli>, there will be no 
good indication that this is so. 

c. Selecting Instructional Methods. Al- 
though it is necessary to"^3Rcu§s TetmroTrTrf' in- 
structional methods and selecUon of instructional 
media separately (as will be done in the next few 
paragraphs), for all applications methods and 
media cannot be considered separately. The only 
meaningful question is. which methods/ media 
combinations are the most effective for given in- 
structional purposes? 

(1) Types of learhihg. After the instruc- 
tional objectives have been specified, the essential 
knowledges, skills, and attitudes to be learned in 
order to achieve each objective must be identified. 
When these have been identified, it is necessary 
to select the instructional methods and media to 
be used in teaching them. One technique for dping 
this IS to relate the behaviors involved to one or 
more of the types of learning. (An instructional 
objective can involve more than one type of learn- 
ing. ) The types of learning, with examples of each, 
are listed in table 5-1. The first six types of learn- 
ing listed in this table have to do with knoivledges, 
and have been arranged in a hierarchy beginning 
with the least complex (association). The next 
higher type of learning is a chain. Chains are 
simply two or more associations linked together. 
Each of the higher types of learning includes all 
of the lower types. 

(2) Instructional methods. Various instruc- 
tional methods are listed and defined in table 5-2 
(17). In identifying the instructional meihod(s) 
to use to teach a particular 'skill, knowledge, or 
attitude, the type of learning (table 5-1) must 
be considered. Probabl>, this will still leave several 
possible alternatives. Considering student differ- 
ences and availability of time and resources will 
^etermine which alternative is most appropriate. 

(a) Student Individual Differences. If the 
planned instruction is to accommodate mdividual 
differences, generally it will permit self-pacing. It 
may also provide for differences in background 
and experience, and even for differences in learning 
styles. Such planning rules out" some of the in- 
'structional method options. 



(h) Individualized Instruction. Individ- 
ualized instruction recognizes that different stu- 
dents have different abilities and needs, and is de- 
signed to provide the flexibilit) to allow for at 
least some of these differences. Entering students 
do not all have the same prior experience, the same 
prior knowledge, of the subject, the same Jearning 
rate, or even the same "best" way to learn. There 
are varying degrees of individualization depending 
on how many of these variables the instructional 
designer tries to provide for. For example, self- 
pacing provides individualization for varying learn- 
ing rates. Individualized instrtiction can assui^e 
one of several different forms depending on 
whedier the course or the learner determines what 
objectives are to be attained, and what learning 
activities will be provided to aid the leanier in 
attaining the objectives. These different forms of 
individualized instruction are represented in 
figure 5-2. 

(c) Time and Resources. A\ailability of 
qualified instructors, suitable facilities, equipment, 
and materials in reference to time allotted for 
instruction, student load, and class size reflect cost 
of instruction. These factors will directly affect 
instructional method selection Although the basic 
criteria for selecting a method should be instruc- 
tional effectiveness, the decision to select among 
equall) effectij^e methods must be made on the 
basis of cost. - — 

d. Instructional Media. Instructional media 
refers to the means used to present information to 
the student. Examples would range from a class- 
room instructor to a book, a flight simulator, a 
computer terminal, prenarrated slides, end items 
of equipment, or television. Properly selected in- 
structional media suppl) the information neces- 
sar> for learning in a form which can be used by 
the student. Instructional media provide some of 
the physical circumstances for proper initial learn- ^ 
ing and for subsequent practice of the knowledge, 
skill, or attituSe to be gained. The necessary in- 
formation for helping students learn can be given 
through a wide variety o£ instructional media and 
combinations of instructional media. The pro- 
cedure or strategy for achieving a training ob- 
jective combines three elemehts in varying ways, 
presentation of information, application or prac- 
tice, and an evaluation which provides feedback 
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fab/eSW. Types of Learning 



TYPE 



Forming 
Associations 



PERFORMANCES RELATED TO DIFFERENT TYPES OF LEARNING 



Involves associating, naming, or responding to a specific input (stimulus). The person 
associates the response with a specific Input only. The response may be vocal, sub- 
vocal (say-it-to-yourself), written, or motor. 

Examples: Naming objects you see; stopping at a red traffic light. 



Forming 
Chains 



Making ^' 
Discriminations 



Involves recalling sequences of actions or procedures which must be recalled in a 
specific order. In a chain the response to one input becomes the input to the next 
response. May involve chains of verbal responses or chains of motor responses. 
Examples: Verbal chain: reciting a memorized poem; stating a rule. 

Motor chain: tying a shoelace; starting an aircraft engine. 



Making 
Classifications 



Using Rules 



Involves making different responses to the various members of a particular class? being 
able to distinguish among input Information sources and/or types and then to respond 
appropriately to each. 

Example, Recognizing the differences among similar guages on an instrument panel 
and reacting appropriately with a vocal, subvocal, written, or motor response. 

Involves responding in a single way to all members of a particular class of observable 
or abstract events. This involves recognizing the essential similarity among a class of 
objects, people, events or abstractions, and recpgnizing the differences which separate 
those objects, people, events, or abstractions which are not members of the class. 
Example: Classifying aircraft as being tactical, fighter, transport, etc. 

Involves applying rules to a given situation or condition by responding to a class of 
inputs with a class of actions. A rule states the particular relationship between two or 
more simpler concepts. It is helpful to think of rules as "if-then" statements. 
Example: If d metal rod is heated, then it will expand. 



Problem 
Solving 



Performing 
Skilled 
Motor Acts 



Exhibiting 
Desirable 
Attitudes and 
Opinions 



Involves comparing previously learned rules to create a higher order rule. 

Example' Troubleshooting a malfunction in an aircraft radar system. (Many rules are 

involved in tracking- down the specific malfunction.) 



Involves manipulative tasks which require the smooth, integrated use of eyes and 
hands. Often this skill entails variations in the actions, where one action will be de- 
. pendent on the results of other actions. ^' 
Examples- Making a sensitive adjustment that requires precise timing; shooting a ntle 
accurately; driving a golf ball. 



Involves enhancing an/mdividuars preference for a particular point of view, idea, or 
course of action. 

Examples A willingness to obey traffic rules in the absence of authority; exercising 
safety practices on the job; desiring to master the skills bein^ taught. 



to the student. Many instructional media can be 
used effectively in each of these elements of the in- 
structional process %vith equal efTectiveness. As 
long as the courseware (such as motion picture 
films, video programs, programmed instruction 
booklets, or prenarrated slides) and the training 
exercise provide all the information the student 
needs at a rate that he can receive — as long as this 



.situation prevails, it does not make any instruc- 
tional difference which instructional medium is 
used. The choice between media of equal instruc- 
tional effectiveness should be based on other con- 
siderations such as cost, maintainability, etc. 

( 1 ) Representative instructional media. . 
The large number of potentially available instruc- 
tional media may be separated into five groups, as 
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Table 5-2. Defin/f/on and Class'tfi cation of /nsfrucf/ona/ Methods 
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METHOD 
Lecture 

Demonstrotion 



Exhibit 



Indirect 
Discourse 



Assigned 
Reading 

Questioning^ 



Programmed 
Questioning 



Student 
Query 



•f 8"g 
e aj 



Seminar 



Discussion 



Practical 
Exercises 



DEFINITION 



A formal or semiformal oral presentation of information by a single indi- 
vrdual; facts, concepts, problems, relationships, rules or principles presented 
orally either directly (as by claSMOom instructor) or indirectly (as by tape re- 
corder, film, or TV^. 



Presentation or portrayal of a sequence of events to show a procedure, tech- 
nique, or operation; frequently combines an oral explanation with the opera- 
tion or handling of systems equipment or material. May be presented directly 
(as by a classroom instructor) or indirectly (as by film, or TV, or by tape re- 
corder if oral orrly). 

A visual or print display used to present information, for example, actual 
equipment, models, mockups, graphic materials, displays, chalkboard, pro- 
jected images, or' sand table. 



Verbal interaction among two or more individuals which is heard by the stu- 
dent, may be a dramatization, such as role playing, or a dialogue between . 
panel members. 

^Printed verbal materials such as books, periodicals, manuals, or handouts. 



A presenter*-controlled interactive process used to emphasize a point, stimu- 
late thinking, keep students alert, check understanding, or review material. 
Questioning may be direct, as by a classroom instructor, or may be designed 
into a film or television presentation. 



A presenters-controller interactive process used to systematically demand a 
sequence of appropriate student responses, may be used directly (as by an 
instructor in a classroom) or indirectly (as by programmed booklets or teach- 
ing machined, including computers).^ 



The provision by which students are given the opportunity to search for infor- 
mation, as byj questioning a classroom instructor, tutor, coach, or an appro- 
priately programmed computer. * 



A peer-controlled group interactive process in which task- or objective-related 
information and experience are evoked from the students. Questions may be 
used to evoke student contributions, but the seminar is distinguished from 
questioning. 

An instructor-controlled interactive process of sharing information and expe* 
riences related to achieving a training objective. 



L 



Student interactions with things, data, or persons, qs is necessary to attain 
training objectives, includes all forms of simulation (for example, games and 
interaction with hardware simulators) and interaction with actual equipment 
or job materials (for example, forms). 



"Presenfer^i instructor and/or courseware. 

NOTE: Other instructional methods not listed are combinations of those given. 
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COURSE DETERMINED 


LEARNER SELECTED 


COURSE 


Individually diagnosed and 


personalized instruction 


DETERMINED 


prescribed instruction 


LEARNER 


Self-directed instruction 


Independent study 


SELECTED 





Adapted from Baird, Belt, Holder and Webb, A 6*hayiaral Approach 
for Teaching Dubuque, lA William C Brawn Co , 1972 



Figure 5-2. Four Forms of IndividuQlized Instruction 



Tabie 5-3. Representative Instructional Media 



INSTRUCTIONAL 
MEDIUM GROUP 



Classroom instructor with instructional aids 
• Classroom instructor 



• Instructional aids 



Multimodal media 



Print 



Peer (or peer group) 



Training devices and simulators 



REPRESENTATIVE EXAMPLES 



Lecturer 
Demonstrator 
Tutor/Coach 
Overhead projector 
Film strip (silent) 
Film slides 
Chalkboard 

Prenarrated slides 
Prenarrated filmstrips 

Slide/workbook/tape recorder combinations 
8mm movies (sound) ^ 
Cassette television 



Books 

Computers (words and numbers only) 
Programmed instruction booklets 
Microfiche 



Role playing 
Discussion groups 
Tutoring /coaching 



^Actual equipment trainers 
Gaming 

Interactive computer (simulation) 
Flight training simulators 
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shown in table 5-3. This table is not intended to 
provide an exhaustive list, but rather is designed 
to show a variety of examples so that the factors 
promoting media effectiveness may be identified 
Descriptions of instructional media as well as ad- 
vantages and disadvantages of each are contained 
in AFM 50-62 and in AFP 50-58, volume IV. 
Handbook for Designers of Instructional Syite7ns, 
Planning, Developing, and Validating Instruction, 

(2) Guidelines for selecting media. 

(a) This portion of the manual provides" 
some guidelines for selecting media. When select- 
ing media, the system designer must consider sev- 
eral factors 

the design of' the specific environment 
for learning, within the context of the overall train- 
ing system constraints (see chapter 3) and the 
instructional objectives (see chapter 4), 

2_ the learning strategy that has been 
planned, based on the types of learning involved. 
The role of instructional media will be to enhance 
the information presentation, the practice, and the 
feedback elements of the lesson. 

^ the extent to which individualization 
of the mstruction would be cost-effective. 

j£ which resources are available for de- 
veloping and producing which kinds of course- 
ware and whether the equipment necessary to pre- 
sent the courseware can be acquired and main- 
tained. 

5 cost-effectiveness. 

(b) The design of student exercises, per- 
formances, or practice items is based on the de- 
termination of what must be done by the student 
to provide satisfactory transfer from the learning 
situation to the job. This*activity may range from 
a student's verbal response in a classroom or filling 
in the blank in a programmed instruction booklet 
to operating a Minuteman Missile Procedures 
Trainer or the C-5 Flight Simulator. Many tasks 

, should be learned under conditions analogous to 
the job, with equipment identical to that found on 
the job However, the practical considerations of 
cost, safety, and resources often prevent the use of 
actual equipment in a job-like environment. 
Further, a well-designed instructional program may 
employ techniques which make the instructional 
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environment physically dissimildi to the job en- 
vironment in order to make the instruction more 
effective. Such techniques include. 

J_ using more feedback during trainmg 
(student is given more knowledge of the results 
of his activities) than is normally provided on 
the job. 

_2 providing many more crises, conflicts, 
equipment breakdowns, or emergencies at closer 
time intervals than is normal to the job environ- 
ment. 

_3 reducing the normal overall opera- 
tioniil time of events to increase the amount of 
practice on critical aspects of selected task per- 
formances. 

.^.providing small amounts of practice 
on selected aspects of similar tasks rather tha;i 
practicing each of these tasks separately, thus in- 
creasing the effective use of limited practice time. 

_5 providing aids, prompts, or guidance 
during learning. Sources of prompting informa- 
tion include aids incorporated into the training 
task or job task, job aids (including checklists, 
picture guides, or other specially designed" aid for 
training), a classroom or field instructor, *a class- 
mate (peer), mediated program (such as an auto- 
mated apprenticeship training approach (18)), 
handouts, and other resources. Actual equipment^ 
may not be designed for optimum training opera- 
tions. The problem of stripped threads in the "re- 
move-and-replace" training exercises which use 
actual equipment represents one very apparent 
aspect of this design problem in training. It is 
probable that any actual equipment designed and 
intended for intermittent operati,on (such as a ^ 
landing gear retracting system) would not stay 
operational when subjected to heavy and almost 
continuous use during the conduct of an efficient 
resident training program. 

(c) An initial step in media selection is to 
determine which level of resemblance to the job 
should be selected for the active learning environ- 
ment. A cross-out matrix (figure 5-3), which lists 
some potential active learning environments, may 
be helpful in making this determination (17). An 
active learning environment will provide the stu- 
dent a working place with some predetermined 
level of resemblance to the job environment. Tasks 
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ACTIVE LEARNING ENVIRONMENT 



ACTUAL ^EQUIPMENT 



FULL SIMULATION 

Actual environment and 

actual response capability 
Physical realism and 

actual response capability 

PARTIAL SIMULATION 
Actual environment 
(physical realism) and 
synthetic/substitute response 



Simplified/modified cues and 
actual response 



Simplified/modified cues and 
synthetic/substitute response 



MOCK-UP, MODEL, 

PART TASK TRXiNER, 
AND VERBAL 



COMPUTER TERMINAL 



CAN USE 


WILL PROVIDE 


ELIMINATE 


TO ACHIEVE 


TRANSFER TO 


BECAUSE OF 


TRAINING 


JOB 


OTHER 


OBJECTIVE 


OBJECTIVE 


CONSTRAINTS 









i ^ 
I 



i 



CARREL 



CLASSROOM 



OTHER 

Figure 5-3. Cross-Out Matrix for Selecting Active Learning Environment (37) 



with concrete information inputs requiring con- 
crefe actions would require a learning environment 
more nearly like the task (for example, land an 
aircraft; remove and install an engine, develop 
efficient reading habits and techniques) Tasks 
with abstract information inputs requiring the pro- 
duction of abstractions are less likel> to require 
- that the training task environment look like the 
actual task environment (for example, compute 
budget estiniates, identify resistors b> color cod- 
ing; develop layout for fabricating replacement 
part) The various possible alternatives, including 
those listed in the matrix, should be systematically 
considered for use. However, it is necessary to 
determine whether the training objective can be 

ERIC ^ 



achieved m the given environment, whether the 
training objective as achieved in the given environ- 
ment will transfer to job performance, and whether 
any other constraints would eliminate a given level 
of resemblance to the job from consideration. To 
use the matrix, analyze the feasibility of using 
each environment. Cross out any level which will 
not provide effective traming or provide transfer 
to the job. -Also reject those active learning en- 
vironments which are impractical for any other 
reason, including budget constraints, need to in- 
terface with an existing program, excessive lead 
time, or commami policy (assuming, of course, 
that the possibility or desirability of relief from 
such constraints has been explored). From the 
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levels of resemblance to the job which remains, 
ensure that all properties needed for the learning 
activit) remain. Determme also if presentation of 
information will be provided at the active learning 
environment or whether another place will be used 
Then give prior it> to the most economical environ- 
ment which permits achieving the training ob- 
jective. 

(d) There is an implication in the pre- 
ceding paragraph that the ideal is always to have 
the instructional situation as similqr as possible 
to the job situation. Generally, this is true, hut 
there are exceptions. Sometimes in the early stages 
of instruction, resetnblance to the job is not de- 
sirable. As an example, consider the student learn- 
ing to recognize different forms of bacteria. He 
could be started off with a microscope and actual 
stained slides — the job environment. But that 
would make the instruction unnecessarily difficult 
because the medium contains much that is distract- 
ing. An alternative is to start the student's in- 
struction ^vith a handviewer and slides showing an 
artist's representation of the bacteria with idealized 
forms and color and no stain debris. As the student 
advances in his training, the slides gradually transl- 
ation to realistic job conditions, and in the final stage 
of his instruction he uses the microscope and actual 
stained slides. 

(e) To assist th e system d esigner in mak- 
ing preliminary decisions, the types df learning 
(table 5-1) are shown in relationship to repre- 
sentative instructional media in tabic 5-4 (17). 
The designer should be able to relate any instruc- 
tional objective to one or more of the types of 
learning, modifying them to include the particular 
learning processes involved. The strategy, placing 
appropriate emphases. op selected elements of the 
instructional process, can be achieved using nu- 
merous media combinations. Jhe matrix can be 
used to eliminate inappropriate options, so that 
realistic trade-offs may be made in the selection of 
instructional media for the training systems. 

(f ) Student differences must be considered 
in the development of an insti*uctional system. 
Differences among students include rate of learn- 
ing and achievement; patterns of behavior, inter- 
ests, motivation, goals, previous knowledge or ex- 
perience, atid readiness; and the capacity to learn 
and apply learning techniques. Individualized in- 



struction ma> be used to accommodate individual 
student differences, reduce redundancy in instruc- 
tion, provide more efficient management, and pro- 
vide learner-centered (learning focused on the 
individual student) performances rather than 
standardized presentation procedures. Self-pacing, 
where the student learns at his own rate, provides 
an effective means of individualizing. While it is 
generally assumed that different studenf aptitudes 
and preferences would require different instruc- 
tional treatments, the techniques fpr detecting and 
using these, differences are not adequately de- 
veloped. Since most of the measures indicating 
general ability differences predict differential rates 
of learning, self-pacing procedures now provide 
the most immediate and largest benefits. The im- 
pact of individualized instruction is most visible in 
the equipment supporting instruction, particularly 
in the selection of instructional media. 

(g) Additional factors must be included 
m the instructional media selection process. These 
factors include the ease of student operation of 
media devices, the level of student control of pres- 
entation rate, and ease of repetition for the stu- 
dents' benefit. Equipment for instruction must be 
eas> to set up and operate. Complex oper^jtions 
in preparation for learning will distract from the 
learning itself, and increase administrative require- 
ments. Various levels of student control of presen- 
-tatioR— ra tes are pos sible— with-4ndividuafe€d-4n- 
struction. ft is recommended that in self-paced, 
mediated instruction, the learner be able to stop 
the program at any point; therefore, a program 
stop feature should be a feature of the selected in- 
structional media. Spoken \¥ords and moving pic- 
tures are transient; if they must be repeated for 
effeqtive individualized instruction, then this capa- 
bility must be acquired in the selected instructional 
media. Features for easy program review and 
timely question/answer (interaction) segments 
should be provided. 

(h) Following decisions about which ac- 
tivities will be performed by the student and which 
information will be provided to the student via 
some media, two important questions remain which 
impact upon media selection. First, how will the 
courseware^ which will be presented to the student 

This manual makes a distinction between "courseware." 
"software," and "hardware." These terms are defined 
in attachment I. 
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be generated? Second, how will the selected media 
devices be obtained and maintained? Since in- 
structional media combinations often can be 
equally effective m presenting the selected infor- 
mation, these two questions may become decisive 
factors. The first of these noninstructional con- 
siderations includes courseware development, 
courseware production, and courseware revision. 
Before selecting an instructional medium, it is nec- 
essary to determine: who (what organization) will 
provide the technical expertise to develop, pro- 
duce, and revise the courseware; and what con- 
tinuing overhead, new facilities, or personnel would 
be required for the courseware development, pro; 
duction, and revision. It does not make sense to 
select a presentation medium for which adequate 
courseware cannot be developed and provided to 
the student. Updating of the courseware should 
be planned for. Experience indicates that a 20 
percent revision rate is an average rate suitable for 
planning when specific related experience data i^ 
not available. 

(i) The other important question concerns 
getting and maintaining the selected media de- 
vices. A critical factor in the selection of instruc- 
tional media hardware is availability. What is 
needed must Be matched with what exists and what 
can be procured. After courseware modes (ob- 
serving, listening, reading, doing) are determined, 
available media hardware should be cfompared and 
selected according to criteria which include usa- 
" bility, reliability, and maintainabilit>. In general, 
it is recommended that a courseware presentation 
format available frorp onl> a single manufacturer 
not be selected. Courseware is more likel> to re- 
main useful if it can be presented on a variety of 
available equipment. 

(j) Usability refers to the suitability and 
case of use by the people who are to use the 
equipment in the place where they are to use it. 
It should also be determined if the place where 
the equipment is to be used can tolerate the po- 
tential heat or noise that the media equipment 
could generateT'^'l^ability refers to the ability of 
all item of equipment to do what it is supposed 
to do when it is supposed to do it. Quantitative 
terms indicating reliability are failure rate (where 
failure is defined as the inability of the equipment 
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to do the job), mean time between tailures and 
success ratio 

(k) Maintainability refers to the level of 
dithculty of work by a qualified maintenance per- 
son required to keep an item ot equipment doing 
what It IS supposed to do for a selected time period. 
Quantitative data is usually based on downtime 
(the time the equipment is not available for use). 
Mean downtime, maximum downtime, maintenance 
cost/operating hour, manhours/overhaul are com- 
mon expressions indicating levels of maintainabil- 
ity. The important factor is whether the equipment 
can be kept operating properly with the avail- 
able resources. Consideration should also be given 
factors such as flexibility in adapting to and in- 
corporating new technology m media hardware. 
Balanced with this should be standardization of 
media equipment without limiting future capa- 
bilities. 

(1) In making decisions on media selec- 
tion, the overall consideration must be, what will 
permit accomplishing the mission with 'maximum 
cost-effectiveness? Each of the media selection 
considerations discussed have a bearing on this 
decision. There are other cost-significant factors 
that must be considered, too, such as: the number 
students who will receive instruction, the phys- 
ical concentration of the students (thai is» as many 
locations across the country, or all at one site), 
the cost of the hardware, the cost of producing 
the courseware, the cost of operating and main- 
taining the hardware, and the cost. "of keeping the 
courseware current. For some expensive media, 
such as television and computer assisted or man- 
aged instruction, an alternative to purchasing a 
complete installation is adding on to an existing 
system.. For example, if a television production 
and distribution system is installed and available, 
adding television sets to the distribution system 
may be cost-effective, whereas buying and installing 
a complete television production and distribution 
system may not be. 

5-3. Determining Resource Requirements and 
Costs. Another activity to be accomplished, during 
this step is*the identification of total resource re- 
quirements. As was stated in chapter 3, it is not 
possible to identify th^ tdtal resources needed 
until some of the activities associated with de- 
veloping the instructional system have been^ac- 
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TYPES OF RESOURCES AND COST FACTORS 



RESOURCES 


COST FACTORS 


• EQUIPMENT 


• DEVELOPMENT 


instructional 




Support 


• liNYtolAAcNI 


Equipment 


• FACILITIES 


Facilities 


Classrooms 




' Laboratories 


• OPERATION AND MAINTENANCE 


Spedal-Puripose 


Equipment , 
Facilities 


• MANPOWER 


Pay and Allowances 


Instructors 


Instructional Administration and 




Supervisors 




Base Administration and Support 





Figure 5-4. Typical Resource and Cost factors 



complished, Leadtime requirements, however, ma> 
make it necessary to identify some resource re- 
quireme^nts prior to finalizmg these activities. The 
specific activities which need to be accomplished 
before totsW resource requirements are established 
include: t™ derivation of learning objectives, the 
identificatioi of instructional media to be used, 
and the detVrmination of instructional sequVce. 
The results of ^these activities should provide the 
information needed to determine total resources. 

' a. Types of Resources and Costs. Several 
types of resources are required to conduct instruc- 
tional programs. If instruction is to reach it^maxi- 
mum effectiveness, resources of the right type and 
quantity must be available on the course start date. 
Figure 5-4 identifies the types of resources and 
cost factors that must be considered in developing 
the instructional system. The following paragraphs 
provide a brief description of these resources. 

(1) Equipment. Two types of equipment are 
needed to support the instructional system. Instruc- 
tional equipment is any item or combination of 
items used for instruction, whether or not it was 
specifically designed for that purpose. The func- 
tion of instructional equipment is to help bring 
about a desired change in the students' performance 
capability. Support equipment includes any item 
or combination of items needed to sustain the in- 
structional system. Items classified as support 
equipment are not used as vehicles of insrf-uction. 



Some examples of support equipment are. chairs, 
desks, typewriters, fihng cabinets, and mobile 
power units. * 

(2) Facilities. An instructional facility is the 
physical complex in which instruction is conducted 
or which provides for the direct^ support of in- 
struction. Instructional facilities may consist of 
classrooms/ laboratories, learning centers, or spe- 
cial-purpose areas. These latter facilities are used 
for the performance of tasks under as near to 
actual conditions as can be simulated within a 
controlled instructional situation. Examples ,of 
special-purpose areas are: aircraft parking and 
runup ramps used by aircraft maintenance courses, 
pole climbing areps used by fixed wire communica- 
tions courses, or site development areas used by 
civil engineering courses. 

(3) Manpower. Manpower is the personnel 
required to accomplish directed missions and work- 
loads. The timely forecasting of future manpower 
requirements is imperative, especially when new 
courses are being developed or existing courses 
are significantly revised. Manpower requirements' 
for the instructional system include: instructors, 
administrative personnel, curriculum specialists, 
writers, and instructor supervisors. 

(4) Costs. Costs are associate^ with each^f 
the above resources. However, costs are considered 
^arately since they represent financial require-T 
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mentJ^. In determining requirements for equipment, 
facilities, and manpower, consideration must be 
given to the cost of developing the resources, the 
investment the> represent, and the cost uf their 
operation and maintenance. Th£ timei) identifica- 
tion and budgeting of these cost factors are just 
as essential to the developmeht of the instructional 
system as the planning for any bf^ the other 
resources. 

b. Selection Criteria: 

(1) Establishing resource requirement!^. The 
identification of resources needed to support in- 
struction is influenced by a number of factors. 
Chief among these are the 

(a) qualitative and quantitative require- 
ments reflected in the instructional standard. 

(b) instructional objectives. 

(c) number of personnel to be qualified. 

(d) time deadlines by which qualified 
personnel are required 

(e) thd manner in which the resources 
arc to be used. 

Other considerations. The above factors 
form the .basic qritena for the selection of re- 
sources Additional considerations are listed in the 
following paragraphs. 

(a) Equipment. Some additional factors 
which need to be considered when determining 
equipment requirements are. 

* Large classes may create a need for 

duplicate equipment in the same classroom. This 
duplication would permit several students or 
groups of students to practice performance tasks 
simultaneously. 

A high student entry rate may also 
necessitate duplication of equipment. 

2^ Facility limitations su^ as power, 
floor area or loading, and air condiopning often 
alfect equipment consideration;?. 

4 T^e equipment ^should provide for 
optimum transfer of learning, and allow for ease 
of operation and maintenapce (19). 

^ The capabilities of instructor and 
maintenance personnel ma> also have an impact 
on equipment selection. 
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(b) Facilities. Additional factors influ- 
encing decisions concerning the number and type 
of facilities needed to support instruction are. the 
nature of the instructional activity, and the quantity 
and t>pe of equipment to be used These factors 
often make it necessary to give special considera- 
tion to: 

J the size and configuration of class- 
rooms, laboratories, and, special-purpose areas. 

2 air conditioning requirements. ' 

_3 power requirements (both electrical, 
and pneumatic) 

_4 ceiling heights and door sizes to per- 
mit entry and positioning of equipment. 

5_ acoustical insulation or baffling to 
suppress noise 

(c) Manpower. The determirt^ation of 
manpower requirements normally involves the use 
of management engineering standards or criteria 
(20). These standards include a definition of the 
work center; the workload factor, its definition, 
and the sources from which the workload was de- 
rived, and guidance necessary for using the stand- 
ard(s). For formal courses, principal factors to be 
checked against these standards are. the averagiK*" 
number of groups instructed during a fiscal quarter, 
and the course stucient load. Other factors which 
have a bearing on manpower requirements include: 

2 student entry rate and group size. 

_2 cojnplexity of tasks to be taught,, 
equipment used to support instruction, and facility 
availability. 

J instructional hours. \^ 
4^ instructor nonavailable time ^- 

(d) Funds. Financial requirements are di- 
rectly related to requirements for equipment, 
facilities, and manpower. These latter require- 
.ments must be translated into unit co^ts. These 
costs become the basis for preparing or revising 
budgets. A further financial consideration is de- 
scribed in the following paragraph. 

(3) Surveying existing resource.^. Prior to 
making a final determination of resource require- 
ments, a survey of existing resources or current 
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assets shpuld be conducted. This surve> Is directed 
toward reducing the quantity of resources which 
have to be procured. 

(a) Equipment and Fdcilitici When sur- 
• veying existing equipment and facilities, the two 
principal considerations, suitabiht> and availa- 
bility, can usuall) be addressed by answering such 
questions as: 

_I_ Does equipment provide for adequatje 
transfer of learning? * 

^ Does the facilit> have the type and 
size of space needed? 

3_ Can equipment or facilities be eco; 



r 



nomically modified? 



Utilized? 



4^ Can equipment or facilities be cross- 



5 -Can-equipment op-t^c4lit4<;&--be-fnad€^ 



( 1 ) Siiivey of existing nuaerials. 

(a) There are times when the writing ef- 
fort can be reduced through the use of suitable 
material which is already available. Before pre- 
paring new material, the availability and suita- 
bility of existing publications and aids should be 
researched Suitability should be based on an 
analysis of the objectives to determine what 
knowledges and behaviors the student should ac- 
quire through the use of the material. If suitable, 
^xisting materials can be used to prevent duplica- 
tion of effort and reduce development costs. Air 
Force activities are encouraged to use off-the- 
shelf publications and aids when these will satisfy 
instructional objectives. 

(b) Commercial self-instructional mate- 
rials are available on a wide variety of subjects. 

.including ^English grammar and composition^ 



available without impairing the mission ot some 
existing program? 

(b) Manpower and Funds. A survey of 
these resources will involve examining current pro- 
grams and authorizations. The purpose of this 
examination is to determine whether or not re- 
sources currently available (or programmed) might 
^be redistributed. The development or revision of an 
, instructional system may relieve some other course 
of all or part of its instructional burden, thus free- 
ing resources for re-allocation. 

5-4. Developing Instructional Materids and Vali- 
dating instruction. Another important activity in 
this step of the model is that of developing the 
most effective instructional materials to support 
each teaching-Uaining activity. These materials in- 
clude such items as texts, studc^ study guides and 
workbooks, instructor guides, slides. tafJe record- 
ings, charts, and case study scenarios 

a. Preparation of Instructjonai. I^afkri- 
ALS. The development of material for instructional 
purposes is a time-con.suming and exacting task no 
matter what type uf media has been .selected. 
Usually, the material takes the form of the printed 
word, either as a textbook, chart, transparency, or 
a programmed text. Even television and film pres- 
entations req^uire the advance preparation of a 
script. Therefore, effective instructional material 
depends upon effective writing. 



virtually all mathematics subjects, clectronicTuncIa^" 
mentals, and many aspects of management. The 
other military services have also developed self- 
instructional materials, many of these in subject 
areas that are common to the Air Force. Other 
publications such as manuals, technical orders, 
and manufacturers' handbooks for specific equip- 
ment should be reviewed for possible application. 
Some of these may require little or no supplemental 
guidanc^ for use by the student, and are espe- 
cially appropriate for instructional use if they 
are the same materials the student will have to 
use on- the job. Often the use of adjunct pro- 
grammed instruction techniques can make these 
materials especially useful for instruction. Air 
Force and comrtiercial self-instructional materials 
and other instructional publications and audio- 
visual packages can be identified through the use 
uf military and commercial indexes, bibliographies, 
and catalogues. ATC/XPTI, Randolph AFB, 
Texas, may be^a^e to provide assistance in identi- 
fying and locating these materials. 

(2) Development of new materials. Through- 
out the development effort, the writer must keep 
in mind the objectives that the material must sup- 
port, and the student for whom the .instructional* 
material is being prepared '* Since the most effective 

''Information concerning the student's background^ read- 
ing ability, and pnor i>no\vledgc ofteji can be obtameJ 
through survey tests (see chapter 4), aptitude tests , such 
as the AQE, and other tests developed by the Human 
Resources Laboratory 
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instruction is studentjccntcrcd, the authur must 
also use techniques consistent with the principles 
for effective learning. The materials must be bo 
constructed that the student will be attentive. His 
attention must be assured b> requiring him to make 
specific responses to the presentation, b> answenhg 
relevant questions, by operating a piece of equip- 
ment, or by performing some other kind of ob- 
servable, measurable behavior. The materials mubt 
move him in the direction of criterion behavior 
and guide him toward master). B> carefully plan- 
ning the stimuli to which the students will.be 
exposed, and by proper application of reinforce- 
ment theory, the desired behaviors can be shaped. 

(a) As far as the Victual writing is con- 
cerned, there are numerous publications, both 
government and commercial, which can aid in 
developing and editing instructional materials. An 

— -anno t^^t^d bibj iogr^^phy of ^gmg thext^ niihlin;^- 
tions is contained in attachment 3. 

(b) In developing instructional materials, 
care must be taken to support the human relations 
concepts to which the Air Force is committed 
Minorities (including women) should be pro- 
portionately represented in visuals and texual ma- 
terials. There should be nothing derogatory or 
demeaning to any group in word or picture. 

(c) The instructional materials that are 
developed should be edited for technical accuracy 
by subject matter expertSr This edit .should not 
be restricted to review of technical content but 
shoujH include such subtle features as adequate use 
of tools, appropriate safety precautions, adequate 
indication of supervision, two-man concepts, and 
iimjor command policies. These materials sho^u[d 
also be edited for composition. This edit must 
ensure compliance with the AFM 5-1 policy and 
essentiality criteria. It should also be concemed 
with the ability of the instruction^ matenaL^ to 
communicate. Do they use language the learner 
understands? Are they free of unnecessary repeti- 
tion and ambiguity? 

b. Validating Instrucuon. At this point in 
the itistructional systehi development process, 
teaching-learning activities to achieve objectives 
derived from job training requirements have been 
developed. Instructional conteqt has been care- 
fuH> sequenced and documented. Appropriate 



niethodb of inbtructiun and media required to pro- 
duce an effective learning environment have been 
^.elected. Instructional materials which will most 
effectivel) support the learning process have been 
developed. Although great effurt has been ex- 
pended, the project cannot be pronounced success- 
tul at this point. It still remains to prove that stu- 
dents instructed" in the s>stem can acquire the 
necessary job skills and knowledges. In develop- 
ing an instructional system, the proof of quality 
can onl> be ascertained through a measurement of 
results in comparison with the specific criterion 
objectives Each step in instructional i)>sicin de- 
velopment can be validated b> checking student 
performance against the performance criteria. This 

"is referred to as system validation. It should not 
be inferred, however, that the complete instruc- 
ItiShal system is developed and then tested all at 
once; rather, portions of it may be tested as they 

~are"produccd. The system is developed umr-by 

unit and is tested or validated on the bases of the * 
criterion objectives prepared for each unit. 

( 1 ) Individual student sampling. Ideally, the 
first step in the validation process, following the 
preparation of a small instructional unit, is to test 
its effectiveness on individual students who are 
representative of the intended target population. 
The students selected for the purpose of vahdation 
should fall within the range af aptitudes, prior 
knowledges, skills, and background displayed by 
the typical input population. Preferabfy, those se- ^ 
lected should possess the higher aptitudes. If they 
have trouble learning the material, -quite obviously 
the material would be too difficult for the less 
capable student. Also, the student having higher 
ability may point out weak areas in instruction. 
Moreover, jf the initial presentation were to he 
' given to students with the lower aptitude, and they 
did well, it would be hard to assess the relative 
difficulty of the lesson. It is simpler to expand ' 
material to make instruction more effective, than 
it is to determine wjiat material to delete in order 
to achieve the optimum level and qHality of in- 
struction. 

fa) Conducting the Individual Tryouf 

During the individual tr>out, a single 
student is closel) observed as he uses the instruc- 
tional material. Wherever the student has difficult) 
or seems uncertain, this should be carefull) noted. 
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even if it concerns onl> one teaching point, if a 
student has trouble understanding or applying one 
rule^or principle, more than likely he will have 
difficulty in accomplishing the behavior to which 
that rule or principle applies. This is especially 
important in that the effectiveness of an instruc- 
tional segment is evaluated onl> on the student's 
ability to perform the behavior, under the condi- 
tions, and within the standard specified by the 
criterion objective. The instrument used to measure 
this performance is the criterion examination for 
the corresponding criterion objective. 

2 During the process of validation, it 
is very important that both the instructional ma- 
terials developer and the student understand their 
roles in the process. It should be explained to the 
student that he is not being evaluated, but the 
system is. He needs to be urged to participate 
lively -^and to respond as required. He should 
also be informed that his responses, with the ex-^^ 
ception of those he makes during the criterion 
ex£imination, will be confirmed so that he will 
be continually aware of his progress. It is through 
the feedback provided b> the student that the 
instructional materials developer can determine 
the adequacy of his presentation and the supporting 
instructional media. During the individual tryout, 
he should carefully note any -problem areas, but 
refrain from providing additional assistance to the 
student unless it is absolutely essential to the stu- 
dent's progress. Even then, the instructional ma- 
o developer should take notes so that he can 
later identify and analyze any areas which may 
reflect an inadequacy m his presentation or de- 
ficiencies in supportmg media. The instructional 
materials developer must hot forget that the failure 
of the student to perform competently at this stage 
is a reflection on the instruction and not on the 
student. 

3^ After the student has completed the 
unit of instruction and the cnterion examination, 
he should be encouraged to discuss any areas in 
which he experienced difficulty. Several individual 
tryouts should be conducted before any significant 
changes are implemented. 

(b) Use of Individual Tryout Results. 
When certain error- patterns occiir during succes- 
sive tryouts, it is an indication that revision is 
necessary. If, for example, several students fail 



to meet the standard of performance established 
for a particular criterion objective, the instruction 
leading to that objective needs to be analyzed 
This analysis plus any supporting comments ac- 
quired during the student interview will probably 
identify problem areas and provide the data upon 
which corrective measures can be initiated. If the 
instructional unit has to be revised significantly, 
it would be best to conduct one or more individ- 
ual tryouts to. check the results of the modification 
before proceeding to small-group testing. How- 
ever, if the' instructional sequence is effective, re- 
quiring only minor changes, a small-group valida- 
tion can be started* 

(2) Small-group tryouts. The reason for 
trying out instruction on a group of students is to 
determine how that portion of the instructional 
system functions under conditions approximating 
_the rirtnn] c|a5;sronm situation. Then too , it is m ore 



economical to gather data concerning the effective- 
ness of instruction from groups than it is from 
lividuals. Therefore, validation efforts are ex- 
pancled to small groups of 6 to 10 students as 
soon a!i satisfactory results are obtained with 
individual tryouts. 

\) Conducting Small-Group Tryouts: 

J_ The first small-group sampling should 
be continued until a total of 20 to 30 typical 
prospective students have been prese^nted the in- 
structional sequence. The students selected for this 
phase of validation should also represent the target 
population. The selections should include, insofar 
as possible, an even distribution of low, average, 
and high aptitude students. 

I. ^^'^ point, *the success of the 

system may have been a result of limiting student 
sampling to those with the higher aptitudes. Now 
it remains to be seen if the same instruction is 
jus> as efficient in teaching students with lesser 
^ility. It should not be surprising if students with 
the lower aptitudes also successfully accomplish 
the^ assigned learnlhg tasks. They may just take , 
longer to complete the sequence. 

3_ At this point in validation, time be- 
comes a factor. It is not sufficient that a student 
learns the material in an instructional sequence, 
he must also complete it within a reasonable period 
of time. In an ideal self-paced system, there are 
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no limits placed on the lengtii of time the students 
can take in achieving objectives. From a practical 
point of view, however, portions of many instruc- 
tional systems will, of necessit>, be group-paced. 
Therefore, efforts to contain a j>egment of instruc- 
tion ^within a realistic period of time should be 
based on requirements and the capabilities of the 
average students. In a group-paced situation, it 
would be just as impractical to pace instruction 
at a rate to satisfy the fast student as it would 
be to meet the needs of the slower student. There- 
fore, during small-group tryouts, the time needed 
by each student to complete an instructional seg- 
ment should be tabulated -in addition to recording 
the accuracy of student responses as in the in- 
dividual tryouts. 

4_ At this time some effort should be 
made to establish the maximum number of trials 
-the stud e nt Avould be- pcmiUc^ to-roac^-CFitcrion 
performance. If proficiency advancement and in- 
dividualized instruction are to be meaningful to 
the student, there must be some described point 
at which remedial training or failure is specified. 
This is particularly true m highly technical and 
flying training programs. Students who are allowed 
continuous trials, when they cannot attain success, 
become, frustrated and agitated. The behavioral 
reactions do not complement the training effort. 
A remediaf program with ax new approach or 
smaller steps mhy well be the alternative. 

(b) Use of Small-Group Try out ^Results: 

/ The small-group tryout should be a 
success because of the nature of the procedures 
for individual tryouts It is probable that any 
changes that Vv^ill have to be made in instruction 
will be minor. As in the individual tryout, if the 
segment of instruction undergoes a significant re- 
Vision, groups of 6 to 10 students should be given 
the modified instruction until additional data have 
been gathered on 20 to 30 more students. This 
cycle of teaching, testing, analyzing, and modify- 
ing is continued until it is pTpven that the students 
can perform to the level specified in the control 
documents. 

^ The small-group tryout results will 
also provide data for planning instructional time 
requirements. This data may necessitate revision 
of equipment and facility requirements and in- 
structor manpower needs. 



Although it IS desirable to conduct 
tryouts before the coursi starts, it is not always 
possible to do^ this. There are situations where 
the students are not available until the course 
starts, and there is no practical way of getting 
substitutes for the students in order to get a 
realistic tryout. 

(3) Operational tryout i^nd implementation. 
The final stage of the validation process is not 
usually conducted by the developers, but by the 
course personnel who would have to operate the 
system. The length of the sequence will vary. 
It may be a portion, a complete block of instruc- 
tion, or a course of instruction in its entirety. 
However, availability of time and resources will, 
to a great extent, dictate the scope of the instruc- 
tional area undergoing a firjal operational tryout. 

There are several reasons why it is 
li^sifa51^"To~COfYa^^ 

out. First, even though the small segments of 
instruction have proven successful, they have only 
been validated in an isolated environment. Now 
the> must be evaluated as integral parts of the 
total system. Second, the analysis of the data 
feedback from this larger sample population will 
provide a solid basis for final revision and refine- 
ment of the system. Third, it will afford oppor- 
tunity to work out administrative, equipment, fa- 
cility, or any other implementation problems which 
may cause trouble later and to update the course 
control documents as required. 

(b) It would be highly erroneous to as- 
sume that once an instructional system is validated 
and implemented, it becomes a finished product. 
There will be problems involving instructor quaJi- 
ficatjons, equipment maintenance, scheduling, and 
variations in student attitude and aptitude. In 
addition, changes in operational requirements (for 
example, changes in the specialty descriptions) 
impose changes in learning objectives and the as- 
sociated equipment and instructional materials. To 
be an effective system, instruction must be con- 
tinuously evaluated and revised to* meet changing 
needs of the student population, operational re- 
quirements, and administrative and management 
problems. 

5-5. Summary. The accomplishment of this step 
in the model requires the careful planning, develop- 
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ing, and validating of in:>tructiun tu ensure that support these activities. The programming of 

the objectives of a course can be effectivel> and equipment, facilities, manpower, and funds must 

efficiently fulfilled. This involves the sequencing bp finalized so that such resources are available 

of teacher-learning activities to produce the best when required. Instructional materials are pre- 

possible learning environment The planning alsu pared, and, finally, all elements of the instructional 

includes the selection of the most appropriate in- system are tested and revised until they are vali- 

structional methodology, equipment, and aids to dated. 
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CHAPTER 



Operation and [valuation of the Instructional 

System 



CONDUCT AND EVALUATE '^^'fw::^,'^ ^ ''"\^T ^'^^^^'^ /^^^ f \ ' '^^uS^^ 



INSTRUCTION 



6-1. Ihtrodudion: 

a. After the instructional program has been de- 
veloped and validated, it is ready to be put mto 
operation. Planning is necessary to ensure that 
instructor and supervisory personnel are adequately 
prepared to conduct and administer the instruc- 
tional program and that supplies and equipment 
will be available when needed. There js a special 
need to prepare the instructors when individualized* 
instruction is used since the role of the instructor 
is changed substantially. 

b. The ISD process does not end when the 
instructional program becomes operatiot\ai. The' 
instructional system must be continuall> evaluated 
to be sure that is is producing qualified graduates 
in the most cost-effective manner. Evaluation pro- 
vides the feedback necessary to ensure that the 
output of the instruction meets job rj^^quirements. 

c. This chapter describes: 



( I ) the changing roles of instructors and stu 
dents; 

' (2) the functions of management in the in- 
structional system; and • 

(3) the process of evaluating the quality of 
instruction and the proficiency of the graduates 
to ensure continued instftictional effectiveness. " 

6-2. Conducting a Student-Centered Instructional 
System. Although many forms of instruction are 
possible in an instructional system, the basic con- 
siderations of the instructional system develop- 
ment process tend to favor student-centered, indi- 
vidualized instruction. This trend is also aided b> 
recent advances in the technoJogy of instruction. 
Modern technology has not solved all instructional 
problems, but the advent of programmed instruc- 
tion, inexpensive tape recorders, slide, filmstrip and 
•film projectors, computer assisted and computer 
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managed instruction, sophisticated aircraft simu- 
rarors,'and'other media and devices, h providing 
an impetus toward more effective, student-centered 
instruction. The obvious consequence of convert- 
ing from an mstructor-centered to a student-cen- 
tered instructional system is the pervading effect 
this has on all aspects of the design of instruction 
(as has been described in the preceding chapters 
of this manual). Another consequence is the altered 
roles of student and instructor. 

a. Student-Instructor Relationship. In a 
student-centered, individualized instructional s>s- 
tem, the instructor becomes a '^manager of learn- 
ing'' and assumes the roles of classroom adminis- 
trator, tutor, and counselor Instead of dealing 
with the class as an entity, he is now able to devote 
much of his attention to the problems and needs 
of the individual student. 

( 1 ) CiassFoonTaUhunisiraior. Despite" so- 
phisticated hardware and innovations, the instruc- 
tor's part in the learning process will never be 
just incidental. There will alvvays be it need for 
personnel to motivate, direct^ guide, and evaluate 
students as they engage in the learning activities. 

(a) Since no two students have the sJnie 
learning patterns or prior knowledge and experi- 
ence, the instructor will be responsible for de- 
veloping individual study plans. He will focus his 
attention on the requirements of the individual, 
rather than the group. This will lead to structuring 

^ or programming subject matter based on the needs 
of the individual student m meeting course ob- 
jectives. The indiv4dual study plans should allow 
for flexibility in scheduling, revision, and consoli- 
dation when group efforts are required. 

(b) The instructor will also ensure that 
instructional materials, equipment, aids, and other 
supplies are readily accessible to the students; 
make certain that the equipment and aids function 
properly; and assist the students as necessary 

(c) The instructor should also make sure 
that each student is progressing according to his 
capabilities within the preplanned scope of the 
teaching -learning activities If student weaknesses 
are discovered through an analysis of evaluation 
data, the student's activities may be rechanneled 
to remedial sequences or the instructor may pro- 
vide individual tutoring. The success of each in- 

ERIC ' 



structional segment should be continuously eval- 
uated as a quality control measure. If deficiences 
in student performance keep recurring, the instruc- 
tional segment involved should be analyzed for pos- 
sible revision. 

(2) Tutor and counselor. From a practical 
point of view, it is almost impossible to design an 
instructional system that will anticipate and pro- 
vide for all the needs of all the students all the 
time. Some students will have trouble meeting 
certain objectives, understanding specific points, 
and performing certain tasks. The trouble spotb 
will be different for different people. The instruc- 
tor must be alert for such problems and be pre- 
pared to provide assistance as needed. The instruc- 
tor may have some students who are exceptionally 
fast or unusually slow. The more capable student 
can be kept from becoming disinterested and bored 
At one of lh> nrce!'e?ar!on techniques -explained in 
chapter 5 is used. The instructor must also diag- 
nose the problems of the less capable student. 
Through observation of student progress and the 
use of diagnostic tests and pretests, the instructor 
can determine how to tailor instruction to meet 
individual needs. This may include providing sup- 
plementary materials or rem^edial instruction. 

(a) Motivation probably contributes more 
to success in learning and to the student s future 
success than any other single factor. It is the 
instructor's responsibility to motivate students. This 
is especially necessary in individualized instruction 
since ihq student is more autonomous. Without a 
desire i6 lear^l, there is little probability that ade- 
quate learning will occur Therefore, the instruc- 
tor must develop techniques which will encourage 
the student to want to learn. One of the best 
ways of stimulating a student toward a greater 
individual effort is to ensure, insofar as possible, 
-that his endeavors meet with success rather than 
failure. Achievement provides intrinsic motiva- 
tion, and tends to strengthen the student's self- 
determination. 

(b) The process of providing guidance or 
of counseling students may be.a new role for some 
instructors, yet in individualized instruction it is a 
most important function. Unfavorable attitudes, 
opinions, or emotions can be detrimental to a 
student's ability to learn. On the other hand, when 
attitudes, opinions, and emotions are favorable, a 
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. healthy climate for learning is established The 
alert instructor can aid the students m overcoming 
their difficulties before the problems seriousl) af- 
fect learning. 

b. Functions OF Man AGEM EN r: 

(1) As a result of the application of steps I 
through 4 of the ISD process, instruction has been 
Carefully designed to shape the job relevant be- 
haviors. No matter how careful this planning, 
the system can break down during implementation 
if management fails to perform its functions. The 
system must have the support of instructors, super- 
visors, and management. There must also be plan- 
ning to supply and maintain the system. It is 
management's responsibility to provide this sup- 
port so that the instructional system can be con- 
ducted as planned. 

ily fnstructors mtist -amierstitnd huw the sys- 
tem IS intended to function and how they are tu 
function in the system This information should 
be provided in an instructor guide prepared by 
the V instructional system designer. Management 
must assure that instructors study and apply these 
instructor guides All personnel concerned must 
understand that arbitrary changes in the validated 
instruction cannot be made without endangering 
the effectiveness of the instruction 

(3) Instructor training (or retraining) may 
also be necessary Instructors faced with the task 
of using new teaching techniques may feel (and 
may be) inadequate Therefore, course adminis- 
trators must ensure that instructors are provided 
with instruction on the application of new con- 
cepts and techniques, as well as the rationale for 
their use. 

(4) Management must also plan for the 
maintenajice of the system Training literature, 
expendable supplies, and tools must be available 
when needed by the -students. Training devices 
must be maintained in functioning condition, and 
a proper level of spares should be available. If 
the organization conducting the instruction does 
not have its own audiovisual equipment, the use 
of loaned equipment must be carefully scheduled 
This kind of planning is always important for eff- 
fective instruction, but it becomes critical when 
self-paced, individualized instruction is involved. 



(5) For a varict> ot reasons, instruction will 
need periodic updating. Revising instructional ma- 
terials requires the same careful planning, develop- 
ment, and validation as developing the original 
instructional system materials. Management must 
plan for adequate, qualified manning to support 
this effort. 

(6) A self-paced instructional system pre- 
sents several special challenges to management. 
The problem of keeping track of the current status 
of each student — what training each student has 
successfully completed, what his special needs are, 
and what equipment is available for him to work 
on next — can be an imposing one. If there are 
many students involved and/or the scheduling is 
especially complex (for example, flying training 
where scheduling is difficult since weather pro- 
vides an unpredictable variable), it may be nec- 
essary te «f,e ^ ^mputer ^he- -j^^eduUng ^ad — 
recordkeeping. In some cases, there is also a need 
for forecasting the gradu^ion date for each stu- 
dent so that he can be promptly outprocessed 
when he has completed the instruction. 

(7) Of course, it does no good to develop 
quality instructional materials and systems unless 
they are used properly and the skills and knowl- 
edges learned by students are applied. This is a 
problem of management of the students who are 
selected to receive the instruction. If the students 
sent to the course do not have the .prerequisites, 
not oril> IS the students* time wasted, but tlje whole 
instructional system design effort can be destroyed. 
The ISD effort is also wasted if the people trained 
are denied the opportunity to apply their skills 
and knowledges. 

(8) Management is also the key to the effec- 
tiveness of the effort to apply ISD in the develop- 
ment of training for new weapon/support systems. 
Unless support of the ISD effort is included in 
the developmental planning for a new weapon/ 
support system, the effort cannot produce infor- 
mation on which to make timely and effective 
instructional decisions. Support signifies such ac- 
tions as making the necessary documentation avail- 
able to the instructional system designers, and hav- 
ing instructional system designers participate in 
the early planning to ensure that major instruc- 
tion and training equipment decisions are based on 
information derived through the ISD process. 
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6-3. Evaluating the Instructional System. 

a. ISD Cycle. The ISD model shows that the 
process is a complete cycle. The feedback from 
evaluation provides control of the instructional sys- 
tem to ensure continumg quality in graduate per- 
formance. If evaluation of the instruction indicates 
a need for change, the cycle is repeated beginnmg 
again with Step 1 of the model. 

b. Validation of Program. Despite the care 
with which the instructional program has been de- 

. veloped, it is still pQssible that the graduate's per- 
formance may not meet the job requirements. 
There may be a number of reasons for this, some 
ofj whjch jnay be related to the instructional pro- 
gram. Validation provides assurance that the in- 
structional program meets the objective-,, but since 
there is a time lapse between when job perform- 

_ mciL requirements were Hptfrmjncd nmi wh^n the 
graduates reach the job, some of the job per- 
formance requirements ma> have changed. An- 
other possible problem is that the tniining nia> 
not have provided sufficient retention. It is also 
possible that there are weaknesses in the instruc- 
tional program, that the mstruction is not being 
conducted as planned, or that students who do 
not have the prerequisites are taking the instruc- 
tion. Through field evaluation and internal eval- 
uation, such problems can be'identificd. 

c. Internal Evai lation. Evaluation provides 
the feedback necessar} to ensure continuing quality 
in graduate performance. The results of internal 

' evaluation do not provide conclusive proof of the 
adequacy or inadequacy of the mstructional sys- 
tem but tliey do* identify some of the system prob- 
lems and suggest arc'as for improvements. This 
review normally includes the following: 

( I ) Cont(ol documents An internal evalua- 
tion must begin with an analysis of the instruc- 
tional system control documents. This should be 
followed b> a ph>sical examination of every com- 
ponent and cver> procedure authorized and, or 
required by these documents The objective is to 
determine if discrepancies exist between the 
planned instructional system as outlined m the 
control documents cinJ what occurs during instruc- 
^ tional system operation 

(2) Resources, Internal evaluation should 
establish whether existing training facilities, equip- 

ErJc 



ment, tools, and supplies are meeting the require- 
ments of the instructional system. In addition, 
support facilities and services should be evaluated 
for adequacy, condition, and maintenance. If de- 
4 ficiencies are *id^tified, recommendation should 
be madeVor crfrective action. 

(3) instructional facilities visits. Visits to in- 
structional facilities must be made in sufficient 
number and be of sufficient duration to assure ob~ 
servation of representative teaching. Trainers, in- 
structional media, and aids specified for use in 
the facilities should be checked for condition, op- 
eration, an4 appropriateness. Instructional litera- 
ture should be checked for availability and quality. 

(4) Instructors. The instructors'* activities 
should conform with the specifications m the plan 
of instruction (or syllabus) and the mstructor 

-™g4iide^-4n^ti:uUoi^niu^~-demon&tf<itc-xtppi^pf4tttc^ 
application of instructional system technology. In- 
structors must be able to detect student problems 
and react to their needs. Instructor records must 
be current and show the amounts of inservice and 
special training completed The instructor rating 
sheets completed by the instructor-supervisors 
should be studied and a determination made as to 
the suitability of the corrective action. 

(5) Measurement program. An important 
source of information as to whether the system 
is functioning effectively comes from the instruc- 
tional measurement program. Such a program can. 

(a) inform each student of his progress 
in meeting the objectives of the program; 

(b) ^ e.stabhsh a permanent record of each 
student's achievement, and allow the student to 
know what his record is; 

(c) identify the need for a remedial pro- 
gram; 

(d) identify, students who are failing to 
meet the prescribed standards of the course so 
that appropriate action can be taken; 

(e) provide a basis for the recognition 
and utilization of studenh who are out,standing, xind 

(f ) provide feedback data for the purpose 
of establishing a constant quality control check 
on the instructional system. 
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The Last objective. ii> prubabiy the must important, 
in that, it tends to ensure continuing success of 
the system Primarily, instruction is eviiluated in 
terms of student attainmeat or nonattainment of 
learning objectives. Therefore, the measurement 
program should consist of performance examina- 
tions designed to measure student achievement of 
the objectives, as described in chapter 4. 

(6) Analysis, The monitoring of the meas- 
urement program is extremely important in verify- 
ing the quality of instruction. However, it is still 
just one aspect of internal evaluation and should 
not be accomplished to the exclusion of other 
methods, such as visiting classrooms, or checking 
the adequacy of equipment and audiovisual aids. 
Internal evaluation involves an analysjs of M 
of the actions which occur in the operation of an 
instructional system. If the analysis identifies cer- 
tain TnstructTonal^sequences whTcTT^are not nieeT-' 
ing prescribed standards, corrective action can be 
initiated. The corrective action, in addition to 
changes in sequence, may involve reaca^mplishing 
other stepfs in the ISD model. 

(7) Other evaluation. As inipc^rtant as is the 
contribution to quality made by internal evalua- 
tion, it is only one of the two parts to the evalua- 
tion process, field evaluation completes llie picture 

d- Field EvAi.uArioN 

(I) Internal evaluation of all components 
of the instructional system provides a basis for 
determining system effectiveness. But, even if in- 
ternal cv^iluation reveals no fault in planning, de- 
veloping, and conducting the instruction, there is 
no guarantee that the graduates vvill perform satis- 
factorily in the field, Field evaluation is also nec- 
essary The purpose of field evaluation is to de- 
termine the ability of recent gradiMtes to perform 
their assigned tasks to the level of proficiency speci- 
fied in the applicable training standard.' if the 
graduates do rfot need what they were taught, or 
do need instruction they didrt't receive, this infor- 
mation must be fed back to the instructional sys- 
tem designers There nvay be a -need to revise 
the instruction If the graduates cannot perform 

ArR ^O-'^S, I'U'hl LmluuUon <>>/ h>tttm{ School (,huI^ 
iuUi \. establishes a pri)^rani for cv.ilu.ition *)f kitailiuUi's 
i)f formal courses and career Jevclupnienl ciuuscs. aiul 
prescribes tfie forms to be used 
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some tasks sufficiently vvelL that may also indicate 
ii need to revise instruction. Perhaps there vvasn't 
sufficient pritctice during instruction to assure re- 
tention. 

(2) /rhe matter ot retention ot instruction 
warrants a special comment. In analyzing field 
evaluation findings, indications that the graduates 
have forgotten some of what they were taught 
should be carefully investigated. The instruction 
they received could be at fault, but there are sev- 
eral other possib^causes relating to job condi- 
tions which shoiBf be assessed. The behaviors 
produced during instruction must be maintained 
in the job situation or else they will disappear 
These behaviors vvill not be maintained on the 
job if the motivation does not exist, or if it is 
unrewarding or punishing to the graduate to con- 
tinue to display them - So, to properly evaluate 
TtltT tifiding Ih anlTe' gradUTiTi^tiaTeriTTerai^^ 
struct ion, it is also necessary to know what hap- 
pened to them after they completed instruction. 
Otherwise it is possible to mistakenly invest time, 
equipment, and money in '*beefing-up" instruction 
that IS not at fault (20) 

(3) A field evaluation must be carefully 
designed and executed to assure the collection 
of accurate and valid data. The evaluation may 
be done through the use of mailed questionnaires 
sent to the supervisor, the graduate, or to both 
It may be done through freld visits by an e valuator 
during which the graduate's performance may be 
observed; or the graduate and his slipervisor may 
be interviewed; or both observation and inter- 
view may be conducted Finally, there is the con- 
trolled job .performance evaluation made by ob- 
servation of the graduate^s-pcrforinance over a 
specific period of lime and evaluated and reported 
by his supervisor 

A few examples will iiulKate sonitf ol the nianv pos- 
sible situations 

a Wh.it 1\k yi.iJihite k'ained (Uiring instniVtion, he 
IS not pctniittcJ to apply on tbc jobi 

h Some of the ^taduatL ^ tcllow-vvorkeis. who fuivc 
not recx'jvfd the instruction he goi*, Jecl threatcneil. I o 
preserve their good will toward him. he stops applying 
what he was taught 

c The graduate, who learned how lo Jo the task 
propel ly. finds lh«it on the job otheis. who do .i h.ilfwav 
job with much less efforL receive the same "rewards' 
thai he does. 
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(4) Questionnaires otter the Ica^t expensive 
procedure for field evaluation and ma> be used to 
collect large samples of graduate information. 
However, questionnaires ma> not be the most re- 
liable form gf evaluation. The validity of the data 
obtained through the use of the mailed question- 
naire tends to be proportional to how well it is 
prepared and distributed. Questionnaires should/ 
be tried out before general use to be sure that the^ 
recipient of the questionnaire understands the ques- 
tions in the same way that the designer of the ques- 
tionnaire does. 

(a) Preparation. One of the most im- 
portant features in the development of a question- 
naire is the guidance given for its completion. In- 
structions should be clearl) stated. Nothing should 
be taken for granted. 

.The mam po rt ion o f the quest lonn a i re 

consists of a list of task and knowledge state- 
ments, either extracted or paraphrased, from the 
instructional standard which is the primary con- 
trol document of the instruction being evaluated. 
The supervisor of the graduate may be asked to 
rate him on his ability to perform each task listed 
in the questionnaire. A similar type of question- 
naire could be sent to the graduate asking him to 
indicate how well he thinks the instruction 
equipped him to meet the requirements of his job. 
He could also be asked to indicate which tasks 
he does not perform. 

2 For full utilization of the question- 
naire technique of evaluation, supplemental ''in- 
formation-seeking" questions may be added. Such 
questions may ask for an estimation of the amount 
* ^ of time the graduate devotes to the individual tasks 
or duties within his Air Force Specialty (AFS). 
Other questions may ask about items of equipment, 
materials, or procedures used by the graduate. An 
open-ended question could be included asking for 
suggestions as to how mst ruction^ ,^might be im- 
proved. ' 

(b) Distribution 

Valid results from the mailed ques- 
tionnaire are largely dependent upon the selec- 
tion of the respondents' A representative sam- 
pling of the total number of graduates is essential. 

ERIC 
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For any AFS, there/ma> be variances in job re- 
quire^ments and utilization of graduate:) b> major 
command, geographical location, level of organi- 

jzation, etc. The sample should allow for these 

/ possible sources of variation. 

2^ Proper timing of the distribution is 
also an important factor. Usually, questionnaires 
shatild be mailed to arrive at the base of assign- 
^^jp^nt between 3 and 6 months after graduation. 
If more than 6 months elapse, it may be impos- 
sible to determine whether the graduate learned 
the skill or knowledge from the instruction, or 
acquired it working on the job. On the other 
hand, if the questionnaire arrives too early, the 
graduate may have had time to perform only a few 
of the tasks listed in the questionnaire (2). An 
addressed return envelope should accompany the 
questionnaire so the student or supervisor can re- 
turn the completed form, without review by higher 
authority, to the training system. 

(c) Analysis of Returns. When the ques- 
tionnaires have been returned, the task of evaluat- 
ing the data obtained can begin. It is a process 
of compiling, collating, and analyzing the infor- 
mation from each command. Particular attention 
should be paid to written comments given freely, 
or elicited by the supplementary questions. Cau- 
tion should be exercised in using data from ques- 
tionnaires exhibiting obvious errors, such as the 
halo eflfect (the indiscriminate rating of all items 
positively) and central tendency (rating all items 
in the center of the scale — no discrimination 
made) Therefore, ver) careful consideration of 
the responses is necessary to assure, insofar as 
possible, that the information accurately reflects 
the opinions of the graduates and their supervisors. 
Carefully prepared, properly distributed, objec- 
tively executed, and critically analyzed question- 
naires can f5'rovide constructive information rela- 
tive to the: 

_/ ability of recent graduates to per- 
form the specific ta^yks for which they were pre- 
pared. 

_2 specific nature of instructional^ de- 
ficiencies as .shown by the graduates' performance. 

J[ details of the jobs actually performed 
by the graduates. 

/ 
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4 extent and specific nature of instruc- 
tion not needed by the graduates in meeting re- 
quirements of the using agencies. 

(5) Field eviiluation visits are pcrtormcd by 
personnel from instructional activities who are sent 
to visit a representative sampling of graduates ap- 
proximately 3 to 6 months after their assignment 
on the job. The purpose of the visits is to obtain 
firsthand information on graduate assignment, utili- 
zation, and proficiency on the job, and Lo validate 
information obtained from questionnaires. Perti- 
nent data are gathered through observation and 
interviews with both the graduate and ,his imme- 
diate supervisor (2). 

(6) Although the evaluator (interviewer) is 
primarily concerned with ascertaining the grad- 
uate's proficiency, he is also concerned with how 
the graduate's skills are being utilized, how well 
he is progressing -through on-the-job training, as 
well as other comparative and sfgnificant aspects 
of the total work environment. For this purpose, 
a list of supplemental questions, similar to those 
contained in a mailed i^estiunnairc, should be used 
as a guide while conducting the interview with the 
graduate's supervisor However, the interviewer 
can deviate from the planned sequence and gen- 
erally fit the interview to the responses he obtains 
from the supervisor or graduate An accurate and 
complete 'set of notes should be taken concerning 
the answprs to significant questions, in addition to 
specific or implied comments. 

(7) The third method for field evaluation 
is conducted jointlv by instructional and using ac- 
tivities to determine the extent to which graduates 
meet using activity performance requirements Job 
performance evaluation is conducted \p the work- 
ing environment at representative using command 
bases. In this type of evaluation, the supervisors 
of the selected graduates are required to rate theic 
performance during the first 8 to 10 weeks of the 
graduates' assignment. The supervisor is requested 
to maintain a weekly record of the task or job to 
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be rated, the frequency of performance, a time 
rating, and the equipment used. As in the other 
methods of evaluation, the applicable instructional 
standard provides the criteria on which the rat- 
ings should be based. 

(8) The suj^f^ervisor is requested to submit 
the weekly performance record for analysis to the 
applicable instructional activity Particular atten- 
tion must be accorded those "returns indicating a 
lack of ability on the, part of the graduate to per- 
form specific tasks. ^ . 

(9) At the' conclusion of the performance 
evaluation, a return visit to the using activities 
should be made>-for the purpose of conducting 'a 
crifique on the evaluation, to gather additional 
data, and to clarify any areas not clearly defined 
ip the evaluation returns. 

6-4. Summary: ^ 

a. To assure an effectively operating* instruc- 
tional system, it is not only necessary to develop 
validated, job-referenced instruction but there must 
also be planning and preparation for the imple- 
mentation and maintenance of the instructional sys- 
tem. This will require training for instructors in 
their new roles and orientation for supervisors and 
managers in the supportive actions they must 
provide. 

b. * 'When the system becomes operational, it is 
necessary to conduct interna! and field evaluations. 
The evidence collected during these evaluations 
must be examined to determine if the graduate 
IS. in fact, capable of satisfying job performance 
requirements This infor/i^ation bccumes the feed- 
back to instructional system personnel. The analy- 
sis of data^ obtained through internal and field 
evaluations should point out both strengths and 
weaknesses in the program. Properly conducted 
evaluation assures a steady flow of pertinent and 
timely information for use in maintaining the qual- 
ity and cost-effectiveness of an instructional system. 
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chapter/ . 

Application of ISD to Knowledges and 

Attitudes' 



7-1. introduction. Knowledges and attitudes exi-^t 
to some degree in all instructional systems. It is 
the pwpobc of this chapter to describe how the 
ISD process and principles can be applied to de- 
sign and evaluate instruction for knowledges and 
attitudes, either in a formal course or in a career 
development course (CDC). 

a. For any type of program to which ISD is 
being applied, a base must be established from 
which instruction can be designed and evaluated. 
If what is to be learned cannot be defined, in meas- 
urable terms, it may not be possible to determine 
if it has been accomplished. 

b. In many instructional systems, instruction is 
designed based on specifically defined and observ- 
able job tasks and related performance standards. 
In some ways it is more difficult to obtain this 
kind of information m areas dcamig mainly with 
the learning of knowledges and attitudes. How- 
ever, the requirement still exists, and a basis for 
designing and evaluating instruction still must be 
'identified. Although there are some procedural 

difi^erences in data collection, the basic process and 
principles of ISD are valid for developing instruc- 
tional systems involvi^ig knowledges and attitudes. 

c. Although there are other methodologies for 
designing instructional s>stems for knowledges and 
attitudes, this chapter is limited to a methodology' 
considered one of the easiest to use. 

'For the purpose of this chapter, knowledf^es arc. defined 
as mental abihtics (recalling facts, identifying concepts, 
applying rules, or solving problems, etc.) and attitudes 
as mental states or conditions which affect motivation 
and behavior. 



7-2. Analyze System Requirements. 

a. The first step in any ISD effort is an analysis 
of system requirements. Its purpose is to describe 
the natiire and scope of man's expected role in 
the operational setting. This inforn'iation then 
serves as a starting point for the design and evalijia- 
tion construction. 

b. A system anal^si^,c}^>gned''f(S^*giff[feV the m- 
formatioiT nXit'essary to begin instructional develop- 
ment for knowledges'arKl/or attitudes involves:* 

(1) identification of the system needs; 

(2) determination of instructional goals 
which (if accomplij-hed) are expected to satisfy 
the needs identified; and 

(3) identification ot measurable behaviors 
for each goal which (if accomplished) will be 
accepted as attainment of the goals. 

c. The purpose of ISD is to develop instruction 
which will satisfy the needs of the system when 
instruction is determined to be the appropriate 
solution Needs are generated in many ways, some 
as the result of management directives inherent to 
the system, and others from observed discrepancies 
of desired versus actual behavior. For example, 
a s>stem need for hetfer qualified managers might 
be generated if low meralc an organization was 
attributed to management deriu<nTcieb. Regardless 
of their sources, system needs are the driving 
factor behind the initiation of any ISD effort. 

d. Based on system needs, instructional goals 
are developed. - Instructional goals describe the ex- 
pected outcomes of instruction in terms of a broad 
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to 

or absjtract int<*pt, state, ui condition (7) For 
exan4)le, to improve the managerial skills of Air 
Force personnel could serve as an instructional 
goal. Another instructional goal might be to have 
managers he sensitive to the tweeds and problems 
of tffose they manage ^ ^ * 

4 ' ^ 

e. Once the needs ot the s> stem (job) have been 
expressed as instructioniil goals, these goals are 
analyzed to arrive at measurable behaviors for 
eacfi These measurable bchiiviur.s are sometimes 
called indicator behaviors ThtT appropriate mdi- 
cator behaviors can serve as acceptable evidence 
that the dbsociated goal has been auained. In com- 
parison to the basic ISD model (described in chap- 
ters 1-6 of this manual), these measurable be- 
haviors ^erve the same function as job performance 
requirements.- In identif>ing such behaviors, a 
team of subject matter specialists (SMSs) is often 
consulted and asked to desi:ribe and agree on be- 
haviors which can be measured, and which (if ac- 
complifhed) would be conlidered an indication of 
achievement of the goaKsfi. It is often critical to 
verify these measurable bSkaviors with the using 
agencies and obtain col^curJe^ceJhat if these be- 
haviors are performed, the system needs will be 
satisfied, tt is also possible to further validate 
these measurable behaviors later during the evalua- 
tion of the instructional svstem b> surveving grad- 

• uates to determine the job-usefulness of the be- 
«»haviors acquired during instruction ^ 

f. An example 'of many measurabl^ behaviors 
related to the previously mentioned goal on mana- 
gerial skills might'be to write a long-range manage- 
ment f^lan. It IS the purpose of the instructional 
go<\l analysis to detemiine all of the measurable 
behaviors which a student would be required to 
exhibit in order to satisfy the goal. 'In some cases, 
one or two measurable behaviors might be suffi- 
cient while other cases might ^require many As- 
sume in this case it Wi}s determined through con- 
sultation^ with vSMSs and the using agencies that 
what specifically is meant by the goal to improxe 
management skills of Air Force personnel is that 
they be able to write Jong-range, mid-range, and 
short-nlnge managemcjk plans. If performed, these 

•For an elaboration on the process of idemifying meas- 
urable behaviocs sec Ooul Afiuhst^ by Robert F Magcr 
(7) Another related method of identifying behaviors to 
serve as the basis of instruction js described b> Gron' 
hind (2M 
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three measurable behaviors would constitute at- 
tainment of the goal. In npst^-ases, the analysis 
will not be this simple, however, the process is 
still the same.. 

g In the second example ot a goal, suppose? that 
It was determined that for a manager to be sensi- 
tive to the needs and problems of '"those he man- 
ages he would have to. identify the possible needs 
and problems which could <irise from his em- 
ployees, be able to recognize manifestations of 
those needy<ind problems and prescribe the ap- 
propriate action for each situation. What started 
out to be largely an attiludinal type goal then be-, 
comes much more definable in terms of measur- 
able behaviors which are largely based on knowl- 
edges The attitudinal portion of the goal is still 
present, however, since it is conceivable that a 
manager might know how he is exited to aet 
toward his employees without actnalty-^^fctefr^at 
way To design instructional activities which ac- 
tually do change behaviors of this kind presents 
a real challenge. In some cases, instruction on at- 
titudes must be supported by contujgencies and 
rewards built into tliL job For example, if a man- 
ager is expected to be sensitive to the needs of his 
employees and yet there <ire no rewards or punish- 
ments as the result of appropriate or inappropriate 
actions, then the full worth of instruction may not 
be realized It is important that personnel ex- 
hibiting appropriate bohiiviors «ire rewarded and 
not punished. - , ' 

h In another example, suppose the goal was 
to understand the classic principles of war and 
their interrelationships. After consulting with 
SMSs and using agencies, it is determined that 
what IS really meant by understand the classic 
principles of war and their mterrelationships is to 
recognize operational examples of each principle 
and for given situations, apply the appropriate 
principles. If performed, these two measurable 
behaviors would constitute attainment of the goal. 

1. In any ease, an agreed upon list of measur- 
able performances would serve the same function 
as a list of job performance requirements. ^ 

7-3. Define Instructional Reaulrements: 

a. To determine "what should be taught," each 
measurable behavior (whether it is an indicator 
for an attitudinal or knowledge type goal) is ana- 
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lyzecLto determine what inusl be learned \\\ urdei 
to perform that behavior. This process identifies 
the content from which instruction is to be de- 
veloped and the types u{ learning (tabic '^-h 
which are involved 

b. Returning to the measurable beh.a\jor used 
earlier to develop a lonf^-range tnanagement plan. 
an analysis would reveal that this behavior re- 
quires a student to recognize such a plan before 
he oould write ond* fo identtjy loni^-rani^e nuin- 
agement plans then would be a le«irning require- 
ment for this behavior and be classified as a con- 
cept classification task according to the types of 
learning in table 5-1 Besides determining what 
should be taught, this kind of analysis provides 
valuable information in developing and sequencing 
objectives, tests, and instructional' activities (This 
will be discussed in more detail later. ) 

c. Once the measurable behaviors have been 
analyzed to determine the specific content of in- 
struction, and the types of learning to be acquired 
are identified (problem solving, rule usmg. etc ) 
the "^objectives and tests can be developed and 
sequenced, 

/-'^ 

*7-4. Develop Objectives and Tests: 

a. 'The learning requirements identified for each 
measurable behavior and the associated type of 
learning is used as the basis for writing objectives 
and, tests The classification of the types of leai|iing 
is an important factor since the different types re- 
quire certain conditions to be stated in their ob- 
jectives which impact-on their associated test items. 
The most critical conditions are foe classifying, , 
rule using, problem solving and exhibiting desir- 
able altitudes and opinions as shown in table 7-1. 
Fgr example, the learning requirement to identify 
long-range management plar^s would be classified 
as a concept classification task: according to table 
7-1, an instructional objective for this task should 
read something like the following: Given examples 
of long-range management plans and noncxam- 
ples^ which have not ^ previously been pres?hted: 
correctly identify each ^ 

^Olher learning taxonomies being used sviih varying 
ilcgrees of success inLlude Gagnc (22), Merrill (8). and 
Bloom and Krathwol (23) * 
*A noneXample is defined as anything which would not 
be classified as an example of a.given concept To be 
useful the nonexample should ci)W*yn s/iwlar irrelevant 
properties of the givqn concept bu^lack one or more 
of its critical properties. 
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b The purpose ot the condition to present ex-^ 
amples and nonexamples which have not been pre- 
sented before is to be sure that the student has not 
mereK remembered or memorized specific ex- 
amples and or nonexamples ot the concept during 
instruction. If memorization of specific exam- 
ple(s) of a concept is all that is desired, then the 
task is, and shc^uld be classified as. forming asso- 
ciations. If, however, the student is expected to 
correctly classify an example of a concept in many 
yr all situations, then it is a concept classification 
task and appropriate conditions listed in table 7-1 
should be controlled. 

c The same rationale applies when explaining 
the purpose of the conditions listed for rule using 
and problem solving. If an objettive'js written at 
the rule using or problem solvmg level, it must 
he tested at the same level. To be sure that a stu- 
dent IS not memorizmg the application of a rule 
to only one situation, or in the case of problem 
solving, memorizing the solution lo one specific 
problem, tasks presented in-the test situation should 
not have been previously encountered by the 
studeat. 

d. One way to evaluate the previous objective 
would be to specify exampl«<"lmd nonexamples 
of long-range managemeirt plans to be presented 
to students -for c6rrect identification. If an "in- 
depth understanding'' of the concept is desired, the 
examples and nonexamples ^an be increased in 
their complexity. TJiat is, the nonexamples might 
be^very similar to e>^amples of the concept, differing 
only on one critical, property : or the examples 
themselves mi|ht be made so complex that it would 
be difficult (but not impossible) lo determine if 
they actually contained the critical properties nec- 
essary to make them examples If the objective 
is tested at this complexity, however, the e;cani- 
pies and nonexamples used during instruction 
should also be at the same difficulty level, 

e Ii? ^vfiting an instructional objective for a 
task which requires a vulunttiry act, thus expressing 
an attitude or opinion, it is important to include 
the condition that the students will not know when 
they are^eing evaluated on the desired behavior.' 

Pros-edures desLribed for testing and designing instruc 
tiun fur attitudes or opinions are adapted from Sihitm^ 
inslrucjiunul Strategies and Media A Plate tu Begin b> 
Mernlf and Goodman (24) *\ 
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Type of Learning 



Table 7-1. Critical Conditions for Several Types of Learning 

" r 



Critical conditions to be included in the objectives and 
controlled in the test items. 



Making Classifications j The examples of concepts given should not have previously been presented, 
(classifying conceptsl Also, to control for finer discriminations, nonexamples of the concept should be 
presented which have not previously been 'encountered. 



Rule Using 
Problem Solving 



Exhibiting Desirable 
Attitudes and 
Opinions 



' The situations to which the rule is to be applied should not have previously been 
encountered. 

The specific problepi to be solved should not have previously been encountered. 



A behavior consistent with or opposed to the desired attitude or opinion must be 
presented and the student's choice observed without his being aware and with- 
out cues being provided as to the desired choice. (For three levels of evaluation 
which are possible, see text.) 



For example, an instructiondl objective for ex- 
hibiting an attitude or opinion might be written 
as follows. Without knowing their behavior is be- 
ing observed, cadets will demonstrate respect for 
the flag during retreat b> properl) saluting. 

f. In testing for attitudes or opinions, it is im- 
portant to provide a choice for the student be- 
tween a behavior (s) consistent with and opposed 
to the desired behavior and not to provide Tues 
to the desired choice. 

g. Attitudes or opinions ma> be evaluated at 
three levels to var>ing degrees of success. Level 
one involves expressing acceptance or nonaccept- 
ance of the stated attitude or opinion, level two 
requires the student tu respond how he 'she would 
act in a described situation involving the desired 
attitude or opinion, and level three is an observa- 
tion of the student's behavior m an victual situ<ltion. 

h. The first two levels can be more readii> 
faked than the third. However, it is sometimes 
difficult in an instructional setting to recreate a 
real world situation, and approximations to the 
actual situation must be used. Regardless of the 
level of evaluation, it is crucial that the students 
not know the> are being tested on their attitudes 
or opinions. 

i. Once the objectives and tests are developed, 
the> should be organized and sequenced into units 
and lessons This ma> be accomplished according 



to- the t>pes of learning, actual job sequence, or a 
combination of these After this is done, it is 
often beneficial to administer the tests in the ap- 
propriate sequence to those for whom instruction 
is being designed, and to those considered experts 
in the area, or representative of the desired atti- 
tudes and opinions. This process validates the tests 
and also provides a check on the selection of in- 
structional requirements and, indirectly, the meas- 
urable behaviors, since the instructional require- 
nicpts were derived from them. For example, the 
experts would be expected to pass the tests and 
the target population of the course would not. 
There probabl> will be exceptions^ but if there is 
a substantial difference from the expected trend, 
then further anal>sis is needed on the tests or the 
instructional requirements and measurable be- 
haviors, perhaps even the instructional goal should 
be reexamined. 

7-5. Plan, Develop, and Validate Instruction: 

a. Based on the types of learning associated 
With the objectives, the appropriate instructional 
activities are developed. Ample guidance is pro- 
vided in AFP 50-58, volume IV, chapter 3, 15 
Jul> 1973, tu accomplish this <ictivit> for knowl- 
edges. 

b. One method of designing and developing in- 
struction for the previous classification objective 
on long-range management plans requires that a 
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clear statement or dehnitiuii ot a lung-iangL man- 
agement plan be prepared, the definition should 
specify the critiLa! properties which luusc a plan 
to be classified as being a long-range management 
plan. 

c. Next, a file of examples and none xam pies of 
long-range management plans is compiled These 
examples and nonexamples ma> be selected, basL-J 
on the degree of understanding desired as the re- 
sult of instruction The examples and nonexam- 
ples to be used during mstruction should not ap- 
pear in any of the tests 

d While the example and nonexample file is 
being developed, feedback to the student should 
also be developed to explain exactK wh\ each 
example 'is one" and why the nonexample "isn't 
one/' This information provides the core of the 
instruction and highlights the critical properties 
of the concept which must be learned if the ob- 
jective is to be accomplished Usually, concur- 
rently with development of the previous material, 
the method of instruction and media to be em- 
ployed should be determined For example, the 
method of instruction might be ^elf-pacmg and the 
medium might be a written workbook In the 
situation described, the student ma) be given a 
workbook with a pretest consisting of examples 
and nonexamples of long-range management plans 
If he passes the pretest, he can proceed to another 
objective. If the student chooses not to take the 
pretest or fails it, the WDrkbook would then pre- 
sent him her with the definition of the concept, 
long-range management plans It might even pro- 
vide additional explanatory material on this defini- 
tion, written in different words or in more detail 

€. After reading the definition, the student is 
directed to the examples and nonexamples of the 
concept. If he is not sure why one is an example 
and another one not, further information is pro- 
vided to explain why and highlight the critical 
properties of the concept The student ^ giv^en the 
option of seeing as man> examples and nonex- 
amples as desired (or provided) with the option 
to take the final lest at any time he feels he has 
mastered the concept 

f. The foregoing is onl> one method and many 
variations could be used, both in terms of methods 
of instruction and the media used For example, an 
instructor could provide a lecture designed in a 

ERIC 



7-5 

similar manner, stating the definition ot the con- 
cept ..md then providing examples and nonex- 
amples and entertaining questions concerning them. 
Some student response system, would be well 
suited for this method of programmed lectui'e, 

g. Designing instruction which actually changes 
attitudes or opinions is a very challenging task. 
Following arc some suggestions which may be 
helpful: 

( 1 ) Attitudes and opinions usually require 
a certain amount of knowledge to be learned be- 
fore a student can rationalize a given attitude or 
opinion, therefore, it is important that the pre- 
requisite knowledge be identified and mstruction 
be provided according to the appropriate levels 
of learning (24). 

(2) Human modeling has been used, suc- 
cessfully in changing attitudes or opinions (22). 
This method requires someone whom the student 
respects and can identify with (a human model). 
The human model then actually exhibits the atti- 
tudes or opinions to be acquired by the student. 
This method can be meshed into the instruction 
presented on the prerequisite knowledges. 

(3) Attitudes or opinions may be changed 
by designing role-play or simulation situations in 
which it is more pleasant for the student to adopt 
the desired attitude or opinion than to retain a 
currently held attitude or opinion (24). The un- 

^ pleasant and pleasant experiences Lan be provided 
by group pressure or a reward .system built into 
the situation. 

(4) Group discussions which help the stu- 
dent examine the consistencies and mcon.sistencies 
of currently held attitudes or opinions may cause 
the desir^^hange (24). 

(5) It is important to avoid specific situa- 
tions during instruction that are intended to be used 
m evaluating whether or not the student has 
adopted the desired attitude or opinion; this helps 
alleviate the po.ssibility of a student remembering 
how he ought to act rather than how he feels (24) . 

7-6. Conduct and Evaluate Instruction. Proce- 
dures for evaluation are contained in chapter 6 
of this manual and in AFP 50-58, volume V. A 
continuing system of modification and evaluation 
will include questions si\ch as. ''How well has 
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the course been taught'^" **Are wc accomplishing 
what we intended to do?" and "How can vve im- 
prove the course'^" Finally, "Is the instruction 
satisf>ii>g the s>stcm needs for which it was de- 
signed'*" The answers to this final question, 
which usuallv can best be answered b> the using 
agencies and the monitonng staff elements of HQ 
0SAF, provide the final step in the validation 
process. The information obtained is the infor- 
mation needed to validate the instructional goals 
as previously described. When the instructional 
goals arc modified, based on information such as 
this and information obtained from students and 
instructors, the ISD process is repeated through 
the five steps again, as necessary. 

7-7. Knowledges and Attitudes Encountered in 
Performance-Based Systems. When knowledges 
and, or attitudes are to be learned in perform- 
ance-based instructional systems, the process of 
designing instruction for them is the same as 
previousl) described in this chapter. In per- 
formance-based systems, however, knowledges 
and, or attitudes are identified as the result of 
analyzing observable job performance require- 
ments into their component parts. Whereas, in 



knowledge-based systems, the knowledges and, 
or attitudes are derived from an analysis of the 
instructional goals and their measurable be- 
haviors. Once knowledge and/ or attitudes are 
identilied as being a learning requirement, they 
should be classified as to their t>pe of learning, 
and objectives, tests, and instruction be developed 
in accordance with the procedures previously de- 
scribed in this chapter and in AFP 50-58, vol- 
umes HI and IV. 

7-8. Summary. In conclusion, ISD can be ap- 
plied to knowledge and attitudinal subject matter. 
In comparison to the basic model described in 
chapters 1-6, there are some procedural differ- 
ences in data collection but the same principles 
of ISD are applied^The following summarizes 
the steps involved when appl>ing ISD to know- 
ledges and attitudes, develop instructional goals 
from s>stem needs, analyze instructional goals 
to determine measurable behaviors which, if per- 
formed, will satisf> the goal, anal>ze nieasurable 
behaviors to determine instructional require- 
ments, develop objectives and tests from the in- 
structional requirements, plan, develop and vali- 
date instruction, and conduct and evaluate in- 
struction. 
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CHAPTER 



8 



Air Forte flying Training 



8-1. Introduction. The ISD proeess and model 
have been described in detail m the preceding 
chapters. The purpose of this chapter is to provide 
supplementar) information of interest to personnel 
planning or performing ISD of Hying training 
courses. 

a. ISD com})incs principles of instructional de- 
sign' and management engineering. In applica- 
tion, the management engineering assumes a larger 
role with an mcrease m organizational size and 
complexity ISD of flying training represents such 
a problem. The objectives of this ehapter are to^ 

( 1 ) address those characteri.stics of flying 
training ISD which eause it to differ from other 
ISD efforts; and 

(2) discuss some methods for resolution of 
problems caused by these characteristics 

b. The following listed items arc indicative ut 
those characteristics which make ISD of flving 
training different 

( 1 ) the number and complexity ot job task 
items; 

(2) mission role diversity , 

(3) inherent danger associated with flying 
training; 

(4) the high degree of responsibility placed 
on aircrews for safe mission accomplishment: 

(5) the broad functional interface between 
operations and functional support agencies; 

(6) use of special media devices (for exam- 
ple, aircraft, simulators, instrument trainers, pro- 
cedures trainers, etc.); 



(7) high cost ot equipment resources, 

(8) high operating and maintenance cost; 

( 9 ) training conducted under various weather 
conditions; 

(10) ihe volume of regulatory constraints 
(regulations, manuals, checklists, policy), 

(11) incompatability of civilian skills with 
military flying skills necessitating training in each 
task; and 

(12) the ISO products which must be de- 
veloped (course training standafd, syllabus, etc.). 

e Although It is b> no means a complete list, 
these characteristics are considered to be repre- 
sentative of most flying training courses and should 
serve as an initial point of departure for discussing 
tlying training ISD The management problems 
created b> these characteristics impact on instruc- 
tional design Such impact ma> require substantial 
tradeoffs between learning effectiveness and prac- 
tical working constraints during development and 
implementation.* The . remainder of this chapter 
will deal with those factors which influence suc- 
cessful applications of ISD in flying training cur- 
ricula. 

8-2. Identification of Goafs and Inrtial Planning. 

Establishment of instructional goals and develop- 
ment of a plan to achieve those goals are pre- 
requisites for successful ISD of flying training 
courses 

a Goals. The goals of instruction are to train 
or educate a specified number of individuals to 
perform specific job requirements. Limits may be 



ERIC 



AFM 50-2 



established on the tjmc and icsuuflcs available tu 
develop instruction using ISD methodolog>. Pohcv 
and guidance constraints also tend to restrict the 
designer s latitude for manpower, funding, etc The 
goa\§ for ISD are the criteria against which the 
ISD curriculum is evaluated Goal statements 
should be specific enough to clearly define the goals 
but general enough to avoid unnecessarv design 
constraints which inhibit goal achievement. 

b. ISD Applic aiion Pi.an Goals alone do not 
provide an adequate base from which to accom- 
plish ISD. A plan must be devebped and co- 
ordinated to ensure that orchestration of effort is 
achieved wi|h order and reason but without un- 
necessary duplication or oversight At unit level 
a formal plan may be unnecessary. At higher 
levels (MAJCOM, air division, etc.) a tormal plan 
is essential in order to establish the authority, task- 
ing, working relationships, etc. A goud plan for 
ISD may contain the following items of infor- 
mation. 

( 1 ) purpose ot the plan, 

(2) ISD goals and constraints; 

(3) directing authority, 

(4) concept of operations: 

(5) tasking (including establishment ot any 
special organizations, stud\ groups, working 
groups, etc. ): 

(6) poiiCN and guidance tor the ISD effort, 

(7) working relationship between tigencics 
(if unasual in nature ) : 

CK) initial milestone completion dates, 

(9) implementation date of the ISD curric- 
ulum; 

(10) products and documentation to be dc- 
livep^as part of the IS!) package (CTS. syllabus, 
data bank, decision rationale, etc ), and' 

(11) training concepts to be employed 

c_JSD ^ Pra< i k \Vo kki\(. ('oss-i kais i s 

Some guiclance is necess.jry regarding ground rules 
for resource usage This guidance should eonie 
directly from the directing authority in writing to 
avoid any misunderstanding The specific type o\ 
guidance which may be provided is beyond t|(ie 
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scope of this chapter, however, a lew examples 
should be sufficient to raise issues relevant to a 
particular ISD effort. 

( 1 ) Manpower 

( a I What manpower ceilings arc imposed'' 

(b) What manpower resources are avail- 
able to draw from? 

(c) What skill levels are required? 

(d) Will training be necessary to meet 
skilled personnel requirements? 

(e) What will be the student load'^ 

(2) Resources Equipment 

(a) What resource equipment utilization 
constraints are imposed'' 

(b) What equipment is available for this 
ISD curriculum? 

(c) Is any new equipment programmed 
lor the future (for example, new aircraft type, 
radar, election seat, advanced simulator, etc )? 

(d) What funding limits are imposed for 
any additional equipment which may be needed? 

(3) facilities. 

(a) Whiit facilities will be available for 
training use'' 

(b) What new construction or modifica- 
tior^ to existing facilities is pending? 

(c) What funds are available for addi- 
tional needed construction ^modifications? 

(d) Has consideration been. given to sup- 
port ♦facilities (for example, quarters, messing, 
alert, weapons storage, hangar*-, ramp space, etc.)? 

(4) Regulations 

(a) What regulatory documents are ap- 
plicable'' 

( b ) Are these subject to waiver if required 
tor training cfTecliveness' 

( ^ ) PoIk \ Giiidanc c 

(a) What is the basic training philosophy? 

(b) What training concepts are accept- 
able"' Which are not'' 



AFM 50*2 



8-3 



These examples arc onl> u tew uf the quebtion.s 
which should be raised in order to further define 
ISD working parameters It is advantageous tu 
identif) these earlv in the development to avoid 
futile expenditure of time and efTort 

*■ d. Long Leadtimt Items- 

(1) Acquisition of long leadtime items. oUen 
requires premature identifieation of quantitative 
and qualitative resource requirements. The ISD 
process should identif) exactiv what is required in 
terms of numbers of instructors, instructional 
equipment, and facilities Unfortunatelv , the time 
between full knowledge of media resource require- 
ments and the scheduled implementation .of the 
new curriculum may be less than that required 
to obtain essential resource items necessary for 
implementation Dela>ing implementation to ob- 
tain desired media, resources may be impractical 
or even impossible. Sometimes instructional de- 
cisions must be made earlier than desired, and 
based on less than complete information in order 
to ensure availabilitv of required training support 
Items when they are needed. Seldom is this an 
easy task. All relevant information must be care- 
fully weighed, and the final decision based on 
the best infoi'mation available at the time the de- 
cision is made Often this means delaying the 
decision until the last moment which offers reason- 
able assurance of timelv acquisition of needed 
items. Since this point is somewhat elusive, it is 
helpful to establish milestone dates for major stages 
of development, such as initial design, requests 
for proposals (RFI^), source selection, dclivcrv, 
testing; maintenance, supervisory, and instructoi 
training, and courseware validation While all uf 
these may not he necessarv or in this order, theie 
should be enough differentiation at the various 
acquisition stages to allow managerial control 
through management hv exception As these mile- 
stones are passed and the ISO progresses, addi- 
tional system requirements data will surface This 
new data mav be used to revise and update earlier 
system requirements estimates and to initiate ikw 
acquisition or modification requests In this man- 
ner long leadtime items cjevelop concurrently with 
instructional design and allow timelv ISO imple- 
mentation. 

(2) Particular interest should be given to 
items such as buildings, ramp space, runwavs. auto- 
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mated ^data processing, special training devices, 
and manpower requirements.^ The office of pri- 
mary responsibility (OPR) for the type of re- 
source or support required shi)uld be able to pro- 
vide all the necessar> data pertinent to planning, 
procurement, and' or construction of these re- 
sources. For example, the civil engineer should 
provide information for building construction or 
modification, ramp space additions, etc., while 
plans and requirements should be able to provide 
information regarding acquisition of special media 
devices such cis simulators, instrument trainers, 
etc These, examples illustrate the coordinated 
management efTort necessary to obtain resource 
support requirements for ISD of flying training 
courses. 

e. Data Aufomaiion Suim»oki Automated 
data processing (ADP) has many potential ap- 
plications m large-scale flying training ISD efforts. 
It is useful for data handling and simulation model- 
ing during course development as well as for 
student/resource scheduling md administrative rec- 
ords keeping during implementation. The follow- 
ing paragraphs provide a brief description of these 
ADP applications to acquaint the instructional de- 
signer with advantages and disadvantages of each. 

( ! ) Data huncHun^ 

(a) In small ISD efforts, manual data 
handling is usually adequate However, ISD of 
flying training courses involves such a volume of 
task analysis datti that manual data handling may 
not be capable of providing desired support In 
this case. ADP may he the only acceptable alterna- 
tive In practice, task analysis data must be ob- 
served, recorded, sorted, indexed, stored, and 
selectively retrieved in a useful, coherent form. 
ADP can perform mc^t of these operations rou- 
tinel) without man) disadvantages inherent to 
human systems. Since task analysis data is compiled 
early in the ISD process, any ADP services de- 
sired must be identified in time to allow for their 
avciilabilitv when needed, AFM 300-6, AFM 300- 
12, and AFR 300-2 provide information for 
obtaining ADP services 

(b) Desirable ADP characteristics for 
handling task analysis data are a flexible indexing 

In consKlcring nitin power rcqnireiiicnts, the nccJ for 
ISD-traincil personnel and stability of assignment foi 
Mich personnel shuuKI not he overlookeil 
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system and a large storage capacity Reg.trdless of 
the indexing system used, it should be conipjtiblc 
with the ADP equipment available, and it should 
cross-reference task items with the respective ob- 
jective, test, function, equipment, procedure, 
knowledge, or decision to which the> relate A 
well-designed index svstem will allow expansion 
and introduction of new task data and will facili- 
tate commonality analyses during courseware 
development. 

(2) Mocielifiii 

(a) An ADP model tor simulating the 
instructional system may be constructed if suf- 
ficient system data is available SucK a "simula- 
tion model" allows analysis of svstem operation in 
terms -of a defined set of input factors without 
the expense mvolved with actual construction and 
operation of the "real" instructional system U^e 
of the model permits the designer to analyze sys- 
tem resource utilization and cost factors Such 
analysis may not' be practical by any other method 
Successive interactions with various media and/ or 
sequence option inputs yield information which 
leads to the mo;;t cost-effective instructional system 
possible under the defined constraints Though the 
benefits of this kind of modeling are great, there 
are inherent disadvantages well. Simulation 
modeling requires detailed information of instruc- 
tional system constraints, including system com- 
ponent interrelationships These relationships are 
not always known, nar do they always remain 
constant with time, therefore, it is sometimes 
necessary to make assumptions regarding them in 
order to "operate" the model. When this is done, 
it should be kept in m)nd that output data reflects 
the inaccuracies of any erroneous assumptions, 

(b) The more closely the model simulates 
the job-world constraints which it portrays, the 
more accurate and reliable the output. Further, 
the expense of identifying and quantifying all the 
instructional system variables and their interrela- 
tionships may be prohibitive. Thus, this kind of 
simulation modeling may not be practical except in 
very large ISD eflforts involving extremely high 
cost resources. The intent here is to identify po- 
tential ADP uses. Their application must be 
tempered with judgment based upon an under- 
standing of the potential benefits and anticipated 
costs in time, manpower, and resources. 
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(3) Scheduling, In large systems, resource 
utilization presents a significant problem. The goal 
is to maximize resource utilization. While a single 
scheduling problem may be manually resolved with 
minimal effort, a large-scale scheduling problem 
involving many students and instructors, media, 
etc , mav make manual scheduling impractical, 
ADP offers the capability to solve most scheduling 
problems quickly and according to specified con- 
straints. Although the computer may be able to 
resolve most schedufing problems, some special 
problems ,for which the computer is not pro- 
grammed may remain. These special problems 
revert to manual scheduling for resolution. ^Pro- 
gramming these special problems may involve com- 
plex constraints which represent costs dispropor- 
tionate to the benefits expected. In summary, ADP 
scheduling is possible. The factors which influence 
the decision are numerous and complex. The bene- 
fits must be weighed against the expected costs 
for ADP services. 

(4) Administrative record keepinv>. 

(a) One of the fallout items from ADP 
scheduling is the data concerning student progress 
and resource utilization. In the case of student 
progress, ADP is capable of keeping realtime 
records of student achievement. With ADP such 
data is quickly available to aid in counseling and/ or 
manual scheduling ADP can relieve the instructor 
of time-consuming "bookkeeping" and permit him 
to devote more attention to instructional rather 
than clerical tasks. 

(b) ADP equipment may be programmed 
to summarize resource utiliaztion in such a way 
that course training managers can identify resource 
excesses, and deficiences This information assists 
Management in justifying and obtaining needed 
additional resources, and allows excesses to be 
employed elsewhere'. Further, it provides an indi- 
cation of maximum student load capability based 
on resource constraints, and identifies additional 
resources necessary if student loads increase be- 
yond programmed levels. 

(^) ADP suinmarv ADP provides capa- 
bilities which enhance ISD development and im- 
plementation, yet, ADP support may be diflficult 
to obtain due to the ex"pense involved and the 
competitive nature of computer time allocation. 



AFM 50-2 



Requirements must be identified early and suf- 
ficient justification provided to confirm ADP sup- 
port needs. Unfortunately, the leadtimc required 
to develop and test computer programs is often in 
advance of full kno\\ ledge of 5r>stem operating con- 
straints. This forces delays for reprogramming and 
further testmg. The ]nslructiun»il designer can aid 
in resolution of this problem b> working close I v 
with the programmer during development of data 
support systems. 

8-3. Media: 

a. Media DEvitts. Flymg trammg courses tra- 
ditionally have employed special types of media 
devices to enhance learnmg effectiveness as well as 
to reduce training costs and exposure to risks in- 
herent m actual flight. Certamly the aircraft is a 
preferred media for criterion performance and may 
be required for much of the instruction Yet often 
the instruction is effectively (sometimes more ef- 
fectively) taught with less expensive devices. Often 
the aircraft is not the ideal training vehicle. Safety 
factors, high operating costs, and an undesirabl^ 
physical and psychological environment (noise, 
uncomfortable temperature, undue stress, etc.) 
often make alternative devices more attractive, in 
terms of cost and or effectiveness. A prime ex- 
ample is a realistic simulator in which hazardous, 
but necessary, procedures may be practiced, re- 
petitively, without risk The designer's task is to 
design the instruction to use the appropriate media 
which will accomplish the necessary training in 
the most cost-effective way. Thus, by selective 
media usage, the mNtructional designer enhances 
both learning effectiveness and economic efficiency. 
Invariably, some tradeoffs will be required during 
media selection; however, the only acceptable 
media will be those which meet instructional ob- 
jectives. From among these, the instructional de- 
signer should select the medium which represents 
the most cost-effective alternatives. More attention 
is devoted to media selection in the following 
paragraphs. 

b SriECTioN OF Mfdia 

( 1 ) Media requirements are determined for 
each objective based upon the medium which 
ofTers the greatest learning tr^msfer capability In 
addition, media altern atives a rc_ identified for com- 
parison and possible tradeoffs between effective^ 



aess and economy. Often it is necessary to sub- 
jectively quantify learning effectiveness for a 
media, objective combination. This value, coupled 
svith estinrates of time to learn the desired in- 
formation or skill using that media, and estimates 
of media utilization costs, yields data regarding 
the cost of teaching a specified objective by various 
media. This process provides a quantified basis for 
media decisions, however, it does not make the 
decision The data obtained from, this type of 
analysis is only as reliable as the learning effective- 
ness, learning time, and cost estimates; therefore, 
managerial experience and other intangible factors 
weigh heavily on the final decision. The question 
which must be addressed is whether the time spent 
in such analysis is worth the potential cost saving 
which may develop from the analysis. While this 
type of analysis offers potentially useful informa- 
tion, the validity of the data may be questioned 
because of its subjective base. Still, it may provide 
the only quantified base for managerial discrimina- 
tion between various media/cost/benefit alterna- 
tives 

(2) When media types have been assigned to 
each objective, the media requirements list should 
be consolidated to identify qualitative and quanti- 
tative needs. A comparison between the consoli- 
dated ISD media requirements list and the list of 
media devices already available identifies items 
which must be procured. Should it be impossible 
(for whatever reason) to procure equipment 
needed, the instructional designer must reevaluate 
the instruction and identify alternative equipment 
(if possible) capable uf supporting instructional 
objectives and related knowledges or skills. 

(3) Often media selection is restricted to 
what is presently available. This is not always 
desirable from an instructional design point of 
view since it may impact directly on learning 
effectiveness; still, it is a realistic economic ap- 
proach, and a common practice. It makes sSnse to 
determine if instructional objectives can be met 
using "on board'] equipment/facilities (including 
those which can be economically modified) before 
investing in expensive new equipment. 

c. Maintenance. Most media devices require 
occasional preventive maintenance, yet, in spite of 
these p^recAUiiQns_iiew_wjlLiiufIei^ break-, 
^dosvn. This should be expected and planned for. 




7o 



8-6 

In some instances, spare equipment ina) be em- 
ployed pending repair or replacement of the mal- 
functioning equipment If the item is expensive, it 
may be impractical to have a spare In this case, 
maintenance and repair service planning should 
allow for '^downtime" during which the equip- 
ment can be worked on without mterfcring with 
instruction 

d Testing 

( 1 ) Most Hying training objectives are tested 
in the aircraft. Others are tested m the media which 
offers assurance that the task could be performed 
in the aircraft, if necessarv. A good example is 
the ejection procedure m many aircraft Perform- 
ing a practice ejection in a real aircraft is unthink- 
able. Thus, for safety and economv considerations, 
the ejection procedure is practiced and the test 
administered in another media, such as a cockpit 
simulator or ejection seat trainer. Naturally, the 
reality of the canopy departing the aircraft, the 
windblast, and the kick of the charged seat may 
not be realistic, but the procedural actions pt'e- 
paratory to actual ejection are identical to those 
required in the aircraft and they are the critical 
aspects which must be learned. There are other 
similar situations such as engine-out approaches, 
engine fire, loss of cabin pressure, etc., which fall 
in this category Simulated tests offer not only a 
safer environment, often, the> .provide a better 
and less expensive learning environment. For 
example, the instructional sequence may be 
stopped at any point during simulation for clari- 
fication or additional practice, as required, then 
re-initiated. Since this can not be done in the 
aircraft (criterion media) for either economy or 
safety reasons, the simulated environment is often 
more effective m certain learning/tcsting situations 

(2 ) As stated previously, the intent in apply- 
ing ISD is to develop quality trainin-g at least cost 
To achieve this goal, the instructional designer 
must carefully weigh advantages and cost of media 
usage for each objective. Selection of the proper 
media for instruction or testing requires answers 
to pertinent questions such as the following. 

(a) What media best suits the objective? 

(b) What alternatives exist? 
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(c) What IS available locally? 

(d) What is the cost of operating and 
maintaining these media'^ 

(e) How long a leadtimc is needed tu 
obtain proposed media? Develop courseware? 
Train instructors, operators, ittatntainers? 

(f) How much instructional time is re- 
quired using the primary media proposed? Sec- 
ondary? 

(g) What cost differential exists between 
media alternatives? 

(h) What are the costs tor manpower, 
supplies, program developmenf^ 

(i) Are the proposed media compatible 
with other courseware with which the media 
may be used? 

(j) Are media (device) repairs/ parts 
available locally? Spare units? 

(k) What storage or special handling re- 
quirements are necessary? 

(I) What type of training is required to 
operate? Maintain? 

(m) Is other equipment necessary to use 
these media*' , 

(n) Where can the objective be tested? 

(0) What tradeoffs are necessary if a 
different medium is used? 

(p) Are safety factors observed in these 

media? 

(q ) Is It realistic'^ , / 

(3) These questions are ,'onl> samples of 
those which must be considpred if the most cost- 
effective media selection is majle. The questions 
will vary based on the working constraints im- 
posed upon media usage. The process becomes 
easier with application and experience since prior 
research regarding media capability, constraints, 
cost, and availability reduce the amount of addi- 
tional work necessary in many (^!^ses. Still each 
medium/objective combination ipu^jt J^e individ- 
ually selected to ensure that the 'i^tructional 
objective is satisfied in the most cost-effective 
manner. 
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8-4. Application of fSD Process: 
a. Impacts: 

( 1 ) Flying traming is a direct mission sup- 
port function. As such, it enjoys a broad func- 
tional area interface with other mission support 
functional areas Consequently, any alterations in 
flight training methods usually affect support re- 
quirements Similarl), changes in functional sup- 
port capabilit) usually result m a corresponding 
change in mission effectiveness. Occjsionall) a 
minor change in either area will produce dramatic 
impacts to mission capability. Thus, it becomes 
essential that all training innovations or support 
capability changes be coordinated in advance to 
avoid adverse impacts to respective functional 
support areas as well as tu the mission effectiveness. 

(2) ' ISD of flying training offers many op- 
portunities to introduce innovative concepts, 
methodologies, etc , which improve training, but 
which also change the character of the course im- 
mensely. The impacts are not always known, and 
even when they are known, their effect on mission 
capability is not always predictable. One solution 
to this problem is to establish an interfunctional 
area working group to assist with recognition and 
resolution of potential impacts. The working group 
is particularly useful to the ISD effort in the initial 
stages of Step I (Analysis of Systems Require 
ments) since group members are able tu^provide 
expertise and known constraints peculiar to re- 
spective functional areas (for example, mainte- 
nance, logistics, personnel, etc. ). This aids in estab- 
lishing ground rules for further ISD. In addition, 
many unanticipated problems occur during im- 
plementation. The working group is able to assist 
in resolution of these problems with minimum im- 
pact since all functional areas are represented 

(3) In practice, the working group must 
play an active role. Their individual and collective 
contrif^utions are essential to successful imple- 
mentation of the ISD curriculum. Working group 
partiwpation often promotes understanding and 
suDjf)ort of ISD and aids the working group repre- 
^^tatives in recognizing and addressing potential 

ERIC 



b. DoCUMbN lAI K)N. 

(1) The survivability of an ISI) curriculum 
IS directly related to the supporting documenta- 
tion. The documentation acts as a baseline for 
evaluation of proposed curriculum changes. To 
be effective, documentation must include all 
pertinent factors which inlluenced a training de- 
cision, including facts, assumptions, majority and 
minority opinion, and the name of the individual 
who actually made the decision. This precludes a 
requirement to perform a second detailed analysis 
of the decision rationale and provides a valid 
comparison between the relative merits of exist- 
ing and proposed traming solutions. 

(2) Documentation of ISD efforts in flying 
training is involved and time-consuming, yet, it is 
essential for ISD curriculum continuity following 
implementation. As an area of recent ISD interest 
and application, little information is available con- 
cerning problems characteristic to ISD of flying i 
trainmg courses. The more documentation that is 
available, the clearer the definition of the types of 
problems which may be anticipated in flying train- 
ing ISD efforts, and the more successful the solu- 
tions to these problems. In this respect, continuity 

of the ISD curriculum is maintained by a self- 
refining process founded on documentation of 
factual ISD data. 

(3) For reasons of ISD iramimi effectiveness^ 
and continuity over the lung term, the importance 
of adequate documentation is so great that one 
cannot afford to ignore it. 

^. Products 

() ) A full ISD effort will result in numerous 
products which have applicability to management, 
course control, and training personnel. The prod- 
ucts which receive the most exposure are those 
directly related to the training for which the effort 
was initially undertaken These may include such 
items as 

(a) job performance requirements. 

(b) trailing requirements 

(c) course control documents. 

(d) courseware: 

J[ instruction guides (lesson plans). 
2 student stud]^ guides. 
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texts). 



etc 



3 student workbooks (programmed 



J learning center instructional packages. 



(e) implementation plan 

(2) In addition to these examples, other 
valuable ISD byproducts may include the follow- 
ing: 

(a) task analysis data bank 

(b) target population data. 

(c) courses which support part or all of 
certain job performance requirements. 

(d) media selection rationale. 

(ej personnel, resources justifications 

(f) ISD documentation 

(3) Products which support an ISD cur- 
riculum should be stated in ISD (behavioral) 
terminology. Su stated, there is a direct relation- 
ship between the curriculum and the product 
which addresses instruction to meet precisely 
stated training requirements. For example, in exist- 
ing course training standard (CTS) terminology 
there is some degree of vagueness. This is not 
conducive to qualit) training pficc it is subject 
to interpretation by both the trainer and user of 



the product CTSs staled in ISD (behavioral) 
t^rms do hot allow the interpretation latitude of 
existing CTSs; therefore, the communication be- 
tween trainer and user is more precise and useful 
as a common ground. This aids the user with 
identification of training deficiencies and the in- 
structional designer (trainer) with resolution of 
training problems which caused these deficiencies. 
Figure 8-1 shows an extract from a ''standard"' 
flying training CTS. Compare this with the ex- 
ample of ISD CTS shown in figure 8-2, and note 
the advantage of the latter 



8-5. Summary, ISD ot Hying training courses uses 
the same process and model as other ISD efforts; 
however, due to the size and complexity of flying 
operations^ certain management problenis assume 
larger proportions. Resolution of these problems 
begins with definition of training goals and the 
development of a plan designed to meet those 
goals. Maximum use of all available management 
tools in a coordinated working environment re- 
duces the ISD problem to more rciilistic and 
manageable proportions. The lessons learned from 
ISD efforts, if propcrl) documented, support ISD 
curriculum continuity and serve to further appli- 
cations of ISD to current and future fi>ing training 
courses. 



d. Formation flying techniques and procedures: 
TWO'SHIP (day & night) 

(1) Ground operations 

(2) Takeoff, climb, leveloff 

THREE & FOUR-SHIP (day only) 



(13) Tdkeoff, climb, leveloff. 



-v-v-v -V -V -v-v-v -V -V 



LEAD 

3 
3 



WING 
3 
3 



Level of proficienc)^ 

"3" Performs In an essentially correct manner. Recognizes and corrects errors. Can apply knowledge 
gained to new or related job elements. Requires normal supervision. 



F/gure8-?. Extract from Course Training Standard (CTS) Using Code Key 
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CHAPTER 7 

Air Force Technical Training 



9-1. Introduction. The iSD concept described m 
the first six chapters of thjs manual applies to 
the development and design of all Air Force edu- 
cation and training programs The purpose of this 
chapter is to relate the process specifically to 
technical traming. Technical training is Air Force 
specialty (AFS) oriented, and this affects the ^ap- 
plication of Steps 1 and 2 of the ISD process, 
therefore, the information in this chapter is limited 
to these two steps. 

9-2. Analyze System Requirements. The applica- 
tion of the first step of the ISD model to technical 
training begins with a general description of the 
operational system and concludes with the docu- 
mentation of specific job tasks and related data. 
The entire analysis is directed toward determining 
the human role in the operational system — a role 
described in terms of duties and tasks necessary 
to perform operational system functions such as 
control, operate, and maintain. 

a. Identification and Organization Proc- 
ess. The process of identifying and organizing 
these duties and tasks requires the accomplishment 
of the following aStivities: 

(1) Identify operational system factors such 
as mission, operational policies and procedures, 
major components or items of equipment, and 
functions. 

(2) Identify job performance requirements 
of the operational system. 

(3) Develop an inventory of valid job duties 
and tasks and supporting data. 

b. Identify Opkraiional Sysfems Factors 
The paragraphs on "SOURCES OF INFORMA- 



TION FOR SYSTEM ANALYSIS" in chapter 2 
provide adequate coverage of this activity. 

c. iDENTpr^JoB Performance Require- 
ments. As explained in chapter 2, the identifica- 
tion of job performance requirements is one of 
the most critical activities in the ISD process, 
since the tasks and related data form the basis 
for the remaining actions in the ISD process. 

( 1 ) Because technical training is Air Force 
specialty oriented, job performance data are col- 
lected and grouped according to the structure .of 
the Military Personnel Classification System .de- 
scribed in AFM 35-1. 

(2) Typically, the identification, organiza- 
tion, and analysis of job performance require- 
ments information is accomplished from ^ the 
general to the specific in an order similar to 
figure 9-1. 

(3) As pJreviously mentioned, the first activ- 
ity in Step 1 of the ISD process is to describe the 
operational system. The second major activity 
is the identification of the job performance require- 
ments. As the major job duties and tasks are identi- 
fied, there is a need to determine which Air Force 
personnel should perform them This involves 
matching the duties with the career field sum- 
maries (a few examples of these summaries are 
shown in figure 9-2 ) and then the duties and tasks 
with the AFSs within those career fields (figure 
9-4 shows an Air Force speqialty summary and 
.specialty description^ Occasionally, some of the 
duties and tasks will not match up with any 

'Destriptjons of the existing Air Force specialties arc 
contained n\ AFM 36-1. for officer personnel, and AFM 
39- 1. for enlisted personnel » 
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OPERATIONAL SYSTEM 
^ (Aircraft) 



FUNCTIONAL AREA 
(Maintenance) 



CAREER FIELD ' 
(Airman Aircraft Maintenance) 



I 



SPECIALTY 
(Aircraft Maintenance Specialist) 




TASK AND KNOWLEDGES 
(Inspects, Cleans, Removes, . . . items) 



(Principles, Theory, etc.] 



Figure 9-?. Sequence of iJenfiftcafion and Analysis 



existing specialty, in which case it ma> be neces- 
sary to propose changes to an existing specialty , 
or creation of a new one. 

(4) After the career fields and Air Force 
specialties involved are determined, the process 
of identifying the specific duties and tasks and 
job supporting data continues. The basic methods 
for collecting the 'required information were ex- 
plained in chapter 2. One of the most compre- 
hensive sources of job data for an existing Air 



Force specialty is the Occupational Survey Report 
developed according to 35-2, Occupational 
Analysis Procedures for Conducting Occupational 
Surveys and Air Force Specialty Evaluation. These 
reports are produced by the Air Force Occupa- 
tional Measurement Center at Lackland AFB, TX. 

An occupational survey report is pub- 
lished after inventories have been administered to 
job incumbents and the data have been key- 
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AIRMAN AIRCRAFT ACCESSORY MAINTENANCE CAREER FIELD (42) 

The Airman Aircraft Accessory Maintenance Career Field includes the skills, func- 
tions, and techniques employed in the maintenance of propellers, pneudraulic systems, In- 
struments, electrical systems, aircrew egress systems, fuel systems, including inflight refuel- 
ing systems, and other accessory systems normally installed in aircraft weapon systems. 
Accessory systems include pneumatic, oxygen, heating, cooling, fire extinguishing, pressuri- 
zation, and fuel air turbine auxiliary test systems. This career field also identifies the spe- ^ 
cializcd skills required in the maiq^tenance of aerospace ground equipment used in direct 
support of aircraft weapon systems. 



Introduction 

AIRMAN AIRCRAFT MAINTENANCE CAREER FIELD (43) 

1. The Airman Aircraft Maintenance Career Field encompasses the mechanical functions 
of aircraft engine installation, removal, maintenance, and repair, and the maintenance, 
repair, and modification of helicopters, turboprop aircraft, reciprocating aircraft, and jet 
aircraft. 

2, The Aircraft Jet Engine subdivision (AFSCs 432X0) includes mstallation, removal, 
maintenance, and repair of turbojet and ramjet engines when installed on missile sNStems 
such as the MGM-13, ADM-20, and the .AGM-28. Also included in this subdivision 
is the maintenance responsibility for small fuel, air turbine auxiliarv engines installed on 
aircraft weapon systems. ^ 



Introduction 

AIRMAN METALWORKING CAREER FIELD (53) 

1, The Airman Metalworking career Field includes the functions involved in fabricating, 
molding, shaping, cutting, and joining metals, and repairing metal parts. This field includes 
aircraft structural repairs, metal heat treating, welding, plating, forging, and machining, 
and installation, modification, and formation of plastic articles. It also includes corrosion 
control for missile, aircraft, and support systems, and technical nondestructive inspection 
of aerospace material parts, components, and pressurized systems. 

2. Excluded from this career field is the corrosion control function associated with Civil 
Engineering areas of responsibility which is included in AFS Protective Coater, AFSC 
552X4. 



Figure 9-2. Career Fields in fAamier)ar)ce Fund/on Evaluation 
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punched and computer analyzed^ A single com- 
prehensive report is prepared for the entire surve> 
sample. However, separate reports can be made 
available for describing particular job types, spe- 
cialties, shredouts, or organizational units. 

(6) Figure 9-3 illustrates entries extracted 
from an actual occupational survey report com- 
piled for the 431X1 career ladder. These extracts 
show the percent of lime spent and the percent of 
members pcrformmg several tasks The task, 'in- 
spect and Service Licjuid Oxygen Systems" is in- 
cluded f)ve times to show differences based on 
aircraft lype and AFS skill level. In figure 9-3. 
a comparison of the 3-skill level personnel shows 
that only 6.32 percent of the apprentice recipro- 
cating engine aircraft mechanics perform this task, 
while 37.58 percenf. of the apprentice jet air- 
craft mechanics perform it The figure also shows 
that the percent of members performing this task 
increases as they go from the 3- to the 5* to the 
7-skill level, but the percent of the members' 
time spent on this task decreases as they go from 
the 3- to the 5- to the 7-skill level.' If the ISD 
process establishes that instruction is required 
on this task, this information has implications for 
the design of the instruction. 

(7) A study of the entire report reveals that 
a large number of aircraft maintenance specialists 
perform this task. This serves to validate the cor- 
responding portion of the specialty description 
(see marked task in figure 9-4). Similarly, other 
statements in the specialty description are either 
validated, or a need for revision is indicated. 

(8) Once the job performance requirements 
have been established, they are used in establish- 
ing, revising, or validating career field descriptions, 
specialty descriptions, training standards, and in- 
structional systems. 

* 

9-3. Define Training Requirements. 

a. Methods for Providing Training. The 
accomplishment of Step 2 of the ISD model re- 
quires the identification of specific training require- 
ments and the selection of methods for providing 
training As explained in chapter 3 of this manual, 
one of the first actions in Step 2 of the ISD process 
is to determine if training is required^ When a 
decision is made to provide training, the next 



action IS to select the most cost-effective method 
or methods. In some situations, a single method 
such as OJT may be completely satisfactory. How- 
ever, more typically, a combination of methods 
such as formal resident training, field training, 
and OJT is needed to meet initial and continuing 
training requirements for both new and existing 
operational systems. In each case the selection is 
based on factors included in the paragraph 
^'CRITERIA AND CONSTRAINTS ATFECT- 
ING THE SELECTION OF METHODS,'' of 
chapter 3. When applying these factors, considera- 
tion should also be given to the following available 
methods of providing training: 

( 1 ) Contract Special Training (Type 1 and 
Contract Teci jnica l Training (Type 6) are formal 
training conducted by civilian industrial or educa- 
tional Institutions. This method of providing train- 
ing is often used to meet initial training require- 
ments of new Air Force systems. Also, contract 
training is used when it is more economical than 
developing an Air Force training capability. An 
example of the latter is the Weathei" Officer Train- 
ing Program yhich is provided by a civilian edu- 
cational instittx^ion under contract with the Air 
Force. 

(2) ATC Special Training (Type 2)^ is 
formal training of ^ne-time or limited nature 
conducted^ by ATC instructors at an Air Force 
base (Or a contractor's location. It is designed to 
qualify experienced personnel in the operation and 
maintenance of weapon, equipment, or space 
systems and related equipment, or in operational 
techniques and procedures. 

(3) Resident Regular Training (Type 3 )2 is 
formal training of a cpntinuing nature conducted 
at an ATC installation. It includes courses designed 
for initial training of personnel, cross-training from 
one AFS to another, training on special or new 
equipment or procedures, and advancement within 
an AFS. 

(4) Field Trainmg (Type 4)- is training 
conducted by ATC personnel at operational loca- 
tions on specific systems and associated direct- 
support equipment for maintenance and aircrew 
personnel. The trainmg may be conducted by 



^Scc AFM 50-5, Volume I, chapter 3, 1 Scpicmber 1974. 
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Description of current job in aircraft maintenance career ladder based on background variables. 
Shows percent members performing and percent time spent on tasks. Tasks are sorted high to low 
percent time spent by group members. 

Task job description, Cases=9233, Tasks=:362, Dutiesi=14, MBRSzr9232 
All airmen in Crr Field DAFSC 431X1X/91 TOTAL SAMPLE 



Cumulative sum of average percent time spent by all members 

Average percent time spent by all members i 

Average percent time spent by members performing 

Percent of members performing 



D-Tsk 


Duty/Task Title 










F 16 
F 17 
M 2 
G 9 


inspect Aircraft for Structural Damage 
Inspect Airframe and Components 
Defuel or Refuel Aircraft 
Launch and Recover Aircraft 


72,89 
70.82 
68.08 
62.31 


1.60 
1.60 
1.64 
1.66 


1.17 
1.13 
1.12 
1.03 


1.17 
2.30 
3.41 
7.69 


1 8 


inspect and Service Liquid Oxygen Systems 


42.67 * 


1.25 


0.53 


42,78 


Task job description, Cases=9233, Tasks=362, Duties— 
DAFSC 43131 A App Reciprocating Engine Aircraft 


14, MBRS=rl74 








1 8 


Inspect and Service Liquid Oxygen Systems 


6.32 


2.24 


0,14 


91,91 


Task job description, Cases:r:9233, Tasks=362, Duties= 
DAFSC 431 3 IE App Jet Aircraft Over Two Engines 


14, MBRS~314 








1 8 


Inspect and Service Liquid Oxygen Systems 


37.58 


1.87 


0.70 


55.98 



I 8 Inspect and Service Liquid Oxygen Systems 



I 8 Insp&rt and Service Liquid Oxygen Systems 
3', 5-, and 7-skill levels 



54.17 



0.85 



Task job description, Caseszr9233, Taskszr362, Dutiesz::14, MBRS— 1607 
DAFSC 431 5 IE Jet Aircraft Over Tv/o Engines Specialist 



47.92 1.22 0.59 A7.4A 



Task job descrij^tion, Cases=9233, Tasks=i362, DutieszzU, MBRSrr672 
DAFSC 431 7 IE Jet Aircraft Over T^o Engines Technician 



0.46 51.79 



Figure 9-3. Exfrocfs from Occupafionaf Survey Report 
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AFSC 43151* 
Semiskilled AFSC 43151* 

AIRMAN AIR FORCE SPECIALTY 
AIRCRAFT MAINTENANCE SPECIALIST 

^ 1. SPECIALTY SUMMARY 

Inspects, repairs, maintains, troubleshoots, services, and modifies aircraft and aircraft installed equip- 
ment; and performs crew chief and maintenance staff fiunctions. 

2. DUTIES AND RESPONSIBILITIES 



a Performs inspections, functional checks, and 
preventive maintenance on aircraft and aircraft in- 
stalled equipment- Performs preflight, post-flight, and 
periodic 'phase inspections of aircraft including 
structures, landing gear, engines, instruments, cock- 
pits, cabins, flight surfaces, and controls Inspects 
and performs functional checks of aircraft systems 
such as hydraulic, electrical, pressurization, lubrica- 
tion, anthicing, vacuum, inWtion and exhaust, and 
installed equipment such as externa! tanks, tow reels, 
hoists, and APUs. Inspects aircraft components for 
Cleanliness, alignment, proper clearance and opera- 
tion, evidence of wear, cracks, and loosene^-s in ac- 
cordance with applicable technical orders. Launches 
and recovers aircraft Inspects and inventories 780 
equipment. 

b Repairs, niainiainy tinJ serutes airtrujt and 
aircraft installed equipment Removes, installs, or 
adjusts aircraft and aircraft system components such 
as control surfaces, wheels, brakes, tires, cawling, 
enclosures, hose, and tubing. Determines and accom- 
plishes maintenance actions required to correct mal- 
functions indicated in aircraft furms and clears forms 
upon completion of maintenance Cleans aircraft and 
engines Identifies corrosion and applies protective 
measures. Troubleshoots malfunctions pertaining to 
aircraft str«iictures. landing gear, control surfaces, in- 



duction, exhaust, ventilation and heating systems. 
Services oil, de-icing, fuel, hydraulic and oxygen 
(gaseous and liquid) systems . Tows and parks air- 
craft and performs engine run-up. Obtains engine 
oil samples (SOAP). Operates ground power equip- 
ment. Operates and maintains nonpowered ground 
equipment. Performs operator maintenance on air- 
craft installed auxiliary power unit. Interprets dia- 
grams and applicable publications and initiates 
technical order deficiency reports (AFTO Form 22). 
Completes maintenance data forms. 

c. Performs aircraft allied functions. Inspects and 
loads tow targets aboard aircraft, assembles and dis- 
assembles glider type targets. Preflights target towmg 
equipment and "^Derforms m-flight operator duties 
associated with launch and recovery of targets. Per- 
forms assigned maintenance in repair and reclama- 
tion of aircraft and the removal, repair, and installa- 
tion of aircraft wheel and tire assemblies. 

d. Supervises aircraft maintenance personnel.^ 
Assigns work and reviews completed work to insure 
compliance with applicable technical publications 
and local procedures. Instructs subordinates in niam- 
tenance of aircraft and aircraft installed equipment. 
Insures compliance with appropriate procedures pre- 
scribed by USAF managenicnt directives. Conducts 
on-the-job training. 



3. SPECIALTY QUALIFICATIONS 



a. Knowledge: 

(1) Knowledge of electrical, hydraulic, and 
mechanical principles as applied to aircrafl, concepts 
and application of AFM 66 — 1. Maintenance Man- 
agement System, maintenance and data reporting, 
and use of diagrams and technical publications is 
mandatory. Possession of mandatory knowledge will 
be determined in accordance with AFM 35— -I. 

(2) Knowledge of supply procedures is de- 
sirable. 

b. Education. Completion of high school with 
courses in physics, hydraulics, and electronics is 
desirable. 
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c. Experience' 

(1 ) Experience in functions such as repair and 
maintenance of aircraft and related installed and 
ground support equipment is mandatory. 

(2) Experience in functions such as performing 
ur supervising aircraft inspections is desirable. 

d. Training. Completion of a basic aircraft main- 
tenance course is desirable. 

e. Other: ^ 

(1) Normal color vision as defined in AFM 
1 60 — 1 is* mandatory. | 

(2) A nwiimum aptitude i^vel of Mechanical 50 
is mandatory. 



Figure 9-4. An Airman Air Force Specialty Description 
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4. SPECIALTY DATA 



a. Grade Spread: 
Sergeant and staff sergeant 
Airman first class 

b. Related D.O.T. Jobs: 
Airplane Mechanic 



43151 
43131 

621.281 



Tire and Tube Repairman 915,884 
The Repairman 9 1 5.884 

c. Related DOD Occupational Subgroup: 600 



Suffix 
A 
C 

F 



5. ^SPECIALTY SHREDOUTS 



Portion of AFS to Which Related 
Reciprocating Engine Aircraft 
Jet Aircraft One and Two Engines 
Jet Aircraft Over Two Engines 
Turbo-Prop Aircraft 



Figure 9-4, An Airman Air Force Specialty Description (Continued) 



either a field training detachment (FTD) or b> 
a mobile training team (MTT), depending on 
cost-effectiveness and practfcalit>. The primar> 
purpose of field training is to qualif> personnel on 
new equipment, new techniques and procedures, 
to acquaint personnel with specific systems, and 
to maintain a given level of proficiency. Mobile 
training teams are normally used to satisfy special 
training requirements when training equipment is 
not available at the training center or when it is 
not desirable to send students to a training center 
Training by mobile training teams is a joint effort 
wherein ATC provides instructors, training ma- 
terials, and funds for the travel and per diem of 
instructors, and the using command provides the 
training equipment- and facilities. 

(5) Command or Other Government Agency 
Training (Type 5)^ is special or regular formal 
training conducted for the USAF by the Army, 
Navy, DOD, or other government agencies, or 
Air Force agencies other than ATC. Recently, 
DOD has designated a number of career fields 
where one service is responsible for providing 
training for personnel of all services. 

(6) On-the-Job Training is a combination 
of self-study and job proficiency development 
through experiences gained on the job. OJT is 

•See AFM 50 5, Volume I, chapter 3, I September 1974. 



basically the responsibility ot the using activity, 
however, the career development courses (CDCsj 
used for self-study are usually developed by the Air 
Training Command A detailed explanation of the 
OJT programs for enlisted personnel is contained 
in AFM 50-23, On-the-Joh Training, 

il). Two other alternatives that shoujd be 
considered in selecting the method for providing 
training are the *"home study" instructional pack- 
age and job performance aids Both alternatives are 
discussed in chapter 3. 

b. Learning Difficulty of Tasks. In ap- 
plying ihe ISD process to apprentice level train- 
ing, when the learning difficulty of tasks is identi- 
fied, the results should be used to assign or 
reassign one of the following three categories to 
the specialty or specialties involved: 

(1) Category A is assigned to specialties in 
which all new airmen must Attend formal tech- 
nical training before assignment to duty in an 
operational unit. The specialty includes a high 
percent of tasks which cannot be learned effective]) 
on-the-job. For example, most aircraft, electronics, 
and missile maintenance specialties are assigned 
to this category due to the learning difficulty of 
the tasks. ' 

(2) Category B is assigned to specialties in 
which only some of the new airn^n attend formal 
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technical training before absignment to dut> m an 
operational unit. The remainmg personnel are 
qualified through on-the-job trammg after assign- 
ment to an operational unit All tasks in these 
specialties can be learned through OJT, however, 
formal training is provided for new airmen bping 
assigned to units located m remote areas or to 
other units where OJT is not feasible or desirable, 
and for other personnel to foster standardized task 
performance in operational units. Typical Category 
B specialties are administration and plumbing. 

(3) Category C is assigned to specialties in 
vvhich new airmen receive no formal technical 
training. All new airmen are qualified through 
OJT after assignment to an operational unit. Ve- 
hicle operators, physical conditioning specialists, 
and painters are all typical specialties currently 
assigned to Category C. 

9-4. Description of Training Standards: 

a. Documenting* Training Requirements. 
After training requirements' are determined, they 
are normally documented in a training standard . 
Examples of training standards currently in use 
are the Course Training Standard (CTS) and the 
Specialty Training Standard (STS). Sample pages 
from these standards are shown in figures 9-5 and 
9-6. 

b. Use of TR/^-i'ning Siandards. Training 
standards do exactly \yhat the title implies They 
establish the standard for .training — a standard in 
terms of job tasks, knowledges, and proficiency 
levels. Also, the training standards are a type of 
Air Force contract. They specify what the training 
activity has agreed to deliver to the using activity — 
individuals with the capability of performing 
specific duties and tasks at a specified level of 
proficiency. 

V 

c. Specialpy Training Standards (STS). 

(1) The STS IS an . Air Force publication 
used to standardize and cpntrol the quality of in- 
dividual training required to achieve the skills 
within an entire Air Force specialty. These skills 
arc based on job performance requirements of 
using activities and documented in terms of tasks, 
knowledges, and proficiency codes. The tasks and 
knowledges listed in STSs correlate with, or expand 
■> on, the specialty description for the Air Force 



spccialt). When it i!^ determined that the duties, 
responsibilities, and specialty qualifications pub- 
lished in AFM 39- 1 , Ainnun Classification Manual, 
are inadequate (partiall) obsolete, incomplete, not 
clear, etc.), a tentative STS should be prepared to 
include tasks and knowledges which are essential 
to current operational requirements. The procedure 
for submitting such standards for approval is de- 
scribed m AFR 8-13. 

(2) A sample page from an STS is shown 
in figure 9-5. The code levels used (proficiency 
code key scale values) are defined in a separate 
code ke) page (not shown). Code levels assigned 
to tasks and knowledges constitute firm commit- 
ments on CDC content and level of training pro- 
vided in applicable ATC formal courses, so it is 
most important that they reflect the training re- 
quirements that were established in Step 2 of the 
I^D process. Because the STS serves multiple 
purposes, and the code ke> must be 'interpreted, 
it is more difficult to prepare an STS that accurately 
describes the training requirements than to prepare 
a CTS that does this. The phrasing of the task and 
knowledge statements, as well as the code levels 
used, must be selected most carefully. 

(3) The coding in columns 2A, 3A, and 
4A uf the STS indicates the minimum proficiency 
recommended for each task or knowledge for quali- 
fication at the 3-, 5-, or 7-skill levels of the Air 
Force specialty. Column 2A also shows the pro- 
ficiency to be attained in the apprentice-level 
course. Proficiency code for the minimum pro- 
ficiency recommended lor the 3-sJcill l^evel AFSC 
and the proficiency attained in the course is the 
same except when dual codes are entered. When 
dual codes are entered, the second code ^hows 
the proficiency attained in the course. Task per- 
fox0iatteo-«5ale value (code 1, 2, ^^t-gilA) defines 
Jevel ot^erformance for ''a specific task state- 
ment. Task know^gdge scale value (code a, b, c, 
or d) defines a level of knowledge for a specific 
task Accordingly, ' code a, b. c, or d may be 
used alone or with a numerical value for a spe- 
cific task statement. Subject knowledge scal^ value 
(code A, B, C, or D) defines a level of knowledge 
for a subject not directly related to any specific 
task, or for a subject common to several tasks. 
.\ccordingly, code A, B, C, or D is used alone 
rather than with a numerical scale value. 
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STS 546XOF 



TASKS. KNOWLEDGES 
AHO STUDY REFERENCES 



AF$C/Cr»« 



PROFICIENCY LEVEL, PROGRESS RECORD AND CERTIFICATION 



3$tciltL«y«l 



0«f. 

OJT 



0*t« C—ttp\9f4 
Initial 



5 Slelll L«v«l 



OJT 



t Tr«ln*«'* 

ivp»r»\fi'* 
ln«H«l» 



7 Skill L«v«l 



0«i* 

OJT 
Starts 



L Jtmin*itt 



19. MITROGEK TUBE BANK TRAILER 

SR:* T0» 00-25-223, 21M-LGM25C-1 , 21M 

a. Operation of cube bank trailer 

b. Connect and disconnect high- 
preaaure hoses 

c. Transfer gaseous nitrogen 

20. MANUAL AI© SAFETY VALVES 

SR: TOa 00-25-223, 35M14-2-41-2, 35M: 
35M14-5-147-3, 35ha4-7-10-3, 35m4-7- 

a. Operation of manual, check, and 
safety valves • 

b. Check manual, check, and safety 
valves for correct operation 

c. Disassemble, inspect, and 
reassemble manual, check, and 
safety vslves 

d. Repair manual, check, and safety 
valves 

e. Acfjust safety valves for popping 
pressure and seal 



-12, 



LGM25C-: 
B 

2b 
2b 



4-3-25- 
6-3, 35<14-7-l 



B 

2b 
2b 

2b /b 
2b /b 



6A9-12-3-2 



35K] 4-3-26-3, 



C 

3c 

3c 

c 

3c 

3c 
3c 
3c 



21. PNEUMATIC OPERATED VALVES 

SR: TOs 00-25-223, 33D9-78-10-1 , 35Kl4-5'-87- 



J, 35MJ4-5 



a. Operation of pneumatic operated 
valves 

bt Check pneumatic operated valves 
for correct operation » 

c. Disassemble, inspect, and 

reassemble pneumatic operated 
valves 



B 
2b 



2b 

2b /b 
2b /b 



22. 



d. Repsir pneumatic operated valves 
e» Adjust pneumatic operated valves 
PRESSURE CONTROL VALVES 

SR: TOs 00-25-223, 35M14-4-17-3, 35MI4-5-93-& 



C 
3c 

3c 
3c 
3c 



C 

4c 
4c 



3-27-3(, 35^4-3-^4-3, 35fa4-3-^-3, 
C 

4c 
4c 

4c I 
o 

4c ^ 

a 

> 

O 

z 

C 

4c 

4c 
4c 
4c 



a. Operation of pressure control 
valves 

b. Check pressure control valves 
for operation 

c. Inspect and disassemble pressure 
control * valves 

d. Repair pressurfr^-^bntrol valves 

e. Adjust pressure control valves 



B 
2b 
.2b 
2b /b 
2b /b 



C 

3c 

3c 

3c 

3c 



/ 



C 

4c 
4c 
4c 
4c 



Figure 9^5. Sampie Page from a Specialty Training Standard (STS) 
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Task. Knowledffe and Proficiency Level 


1. 


CONSTRUCTION FEATURES AND OPERATING PRINCIPLES OF WHITE 
DIESEL ENGINE 




a. 


The design features of model 40SX-6 engine 


B 




b. 


Operating principles of diesel engines 


B 




c. 


Importance of optimum engine performance, and 
factors affecting engine power output 


C 




d. 


Locate and identify engine and system components 


4c , 




e. 


Arrangement and operation of engine system components 


B 


2. 


ENGINE LUBRICATION SYSTEM COMPONENTS 






a. 


Clean and replace oil filters and strainers 


3c 




b. 


Check oil for dilution, cleanliness, and proper grade 


3c 




c. 


Inspect system for leakage 


3c 




d. 


Adjust oil pressure regulating valves 


3c 




e. 


Service engine oil system 


3c 


3. 


ENGINE Alh INTAKE AND EXHAUST SYSTEMS 


\ 




a. 


Cleaning and servicing of air cleaners, filters, 
breathers, and strainers 


4 

B 




b. 


. Inspect turbochargers for operating condition and 
oil seals for leaka&e 

•I* 


3c 




c. 


Inspect and maintain intake and exhaust silencers 
and ducts 


3c 




d. 


Inspect system for leakage and general condition 


3c 


4. 


ENGINE AIR STARTING SYSTEMS 






a. 


Inspect and service air compressor systems 


3c 




b. 


Inspect, clean, and adjust engine air starting components 


3c 



Fig^Jfe9^6. Sample Page frow a Course Training Standard (CIS) for Technical Training 
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d. Course Training Siam>ard (CTS) 

(1) Course training standards are used to 
document the training requirements for a spe- 
cific course of instruction Typicallv. the tasks and 
^ knowledges contained in a CTS reflect unique 
training requirements for a specific weapon or sup- 
port system rather than for a spccialtv. For ex- 
ample, a field t^aining detachment responsible for 
developing a course for aircraft maintenance spe- 
cialists and technicians in support of the C-141 
aircraft would prepare a CTS containing onl> those 
tasks and knowledges needed for that aircraft. In 
another case, the CTS for missile launch officers 
for the LGM-25 ballistic missile program would 
include only those tasks and knowledges required 
for that weapon system Although m both cases 
the training standards are based on training re- 
quirements that relate to job performance require- 
ments, there still is a need to coordinate the train- 
ing standards with the using command(s) One 
reason is that (for a career field course) the data 
obtained during Step I reflect conditions as of 
when the data were obtained. However, the using 
command(s) may have a requirement to change 
ongoing practices. The coordination will pro- 
vide the opportunit) to change the training standard 
to reflect tomorru^\ needs Even with training 



9-ri 

standards for new weapon/ support systems, there 
still may be some constraints the using' com- 
mand(s) must live with which are not compatible 
vvith the version of the traininr; standard sent out 
for coordination. 

(2) Figure 9-6 shows a sample page from 
a CTS for technical training This standard uses 
the same code key as the SIS. It is also possible 
to develop a CTS m which onl) statements are 
used to describe the tasks, knowledges, and pro- 
ficiency levels For an example of such a standard, 
see figure 8-2. 

9-5. Summary. As with other areas ot application 
of ISD,'the application to the design and develop- 
ment of technical training involves some special 
problems and special documentation Most of 
what IS '^different'' about applying the ISD process 
to technical training relates to its being Air Force 
specialty oriented, and affects primarily Steps 1 
and 2 of the model. So, this chapter has dealt 
in more detail with the occupational survey. Air 
Force specialt> descriptions, the -alternative meth- 
ods for providing training, the categorization of 
Air Force specialties with respect to method of 
apprentice-level training, and the training standard. 



By Order of the Secretary of the Air Force 

Official DAVID C. JONES, GeneraU USAF 

Chief of Staff 



JAMES J. SHEPARD, Colonel, USAF 
Director of Administration 



^Summary of Revised, Deleted, or Added Material 



This revision significantly^ changes the firlf six chapters of %e manual. The t«rm "enabling" in relotion to* 
ob)ectJvt$ is dropped, and "enabling ob|tctive test^' has b«en r«nam«d "diagnostic test/' Guidonce if 
expandtd on seUction of instructional methods, ond there is a different approach to media selection "Expio 
nation of Terms" is expanded ond made an attochment, and the bibiiogrophy is exponded and updated 
To facilitate the application of ISD to the full spectrum of education ar\d training, three new chapters have 
been added Chapter 7, "Application of ISD to Knov/ledges and Attitudes", Choker 8, "Air Force Flying 
Training", and Chapter 9, ''Air Force Technical Training." 
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ATTACHMENT 



EXPLANATION Of TERMS 



Action (See Rci>ponse.) 

Algorithm An orileriy procedure or exact 

prescription for solving a problem. In an al- 
gorithm, the physical layout of the presentation 
shows the relationship between input data, rules, 
and outcomes. Algorithms replace continuous 
prose as an instrument for communicating com- 
plex Fules and regulations. (Figures 1-2 through 
i-6, AFP 50-58, Volume I, 15 July 1973, are 
algorithms.) 

Aptitude Test A test or battery of tests de- 
signed to show a person's capacity for a particular 
type of behavior in a single field or m"" several 
related fields. 

Association The connection made between 

an input (stimulus) and an action (response). For 
example, a person responds with the letter ''A'' 
when heiffing the code ''dih-dah.'' 

Automated Apprenticeship Training (AAT) 

-A trailing approach in which the trainee 



♦ Chaining- 



receives tutorial guidance through pictures with 
accompanying sound when provided opportuni ty 
to practice. Trainee performance is evaluated b> 
peer or instructor. 



Behavior- 



-Any activity, overt or covert» ca- 



pable of being measured. 

Behavioral Objective A statement that 

specifies precisely what behavior is to be exhibited, 
the conditions under which behavior will be ac- 
complished, and the minimum standard of accept- 
able performance. 

Carrel (Learning) A study cubicle designed 

for use by one or a small jnumber of students. Car- 
rels- usually contain audiovisual media for pres- 
entation of programmed audiovisual instruction. 



■The linking together ot a series 
of discriminabl<!' responses in a particular order. 
The completion of each response provides the 
stimulus for the next response. May involve 
chains or verbal responses (reciting a list of num- 
bers) or chains of motor responses (following a 
procedure). (See also Association and Discrimi- 
nation.) 

Computer-Assisted Instnictiun (CAl) An 

instructional^ method whereby students interact, 
(usually) individually to instruction presented 
through a variety of media, usually computer con- 
trolled or monitored The student can respond 
to the stimuli in a variety of ways, such as with 
a keyset or pointing device. ' He is immediately 
informed as to the accuracy of his response. The 
computer uses algorithms to diagnose his response 
to determine the probable reason for any errors, 
and to present additional instruction in accordance 
with the diagnosis. 

Computer Directed Training System (CDTS) 

-A training system that involves dependent 



3JLi-bjLysi&gi& (fua£tiuatiL^-u£t^_iiCje_^tQd caur^e^ate) 
and related documentatic^n. Functional software 
enables use of hardware by the course designer 
to code course material and by the student to 
interact with courseware. Courseware is the sub- 
ject matter and instructional data in whatever 
media required by the student to complete his 
course through use of a computer terminal. (See 
also Computer-Assisted Instruction and Com- 
puter-Managed InstructTon.) 

. Computer-Managed Instruction {CMI) In 

CAf the student interacts directly with instruction 
prestitUed by computer controlled or monitored 
equipment. In CMI, the student does not neces- 
sarily interact directly with the equipment, though 
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he may be ^'online'' for testing, diagnosis, and 
prescription. The role of the computer in CM! 
is to aid the instructor in managing the instruc- 
tional program. 

' ConfinnatioiK 



-Notification to the student of 
the correctness of his response. (See also Feed- 
back and Knowledge of Results.) 

Constraints Limiting or restraining condi- 
tions or factors, such as policy considerations, 
time limitations, environmental factors, and budg- 
etary and other resource limitations. 

Cost-Effectiveness A comparative evalua- 
tion derived from analyses of alternatives (actions, 
methods, approaches, equipment, weapon systems, 
support systems, force combinations, etc.) in terms ^ 
of the interrelated influences of cost and effective- 
ness in accomplishing a specific mission or ob- * 
jective. ^ 

\ 

Course Control Documents Specialized 

publications used to control the quality of instruc- 
tion. Specialty training standard, course training 
standard, plan of instruction, syllabus, and course 
chart are course control documents 

Courseware The technical data, textual ma- 
terials, and audiotapes, slides,* movies, TV cas- 
settes,, and other audiovisual instructional mate- 
rials. (See also Hardware and SofKvare.) 

Criterion-Referenced Test (CRT) Test to 

determine if behavior as specified in objectives 
has been acquired. Dunrfg instructional system 
development, the CRT can be usqd to measure 
the effectiveness of the instructional system. May 
— involve-multiple-eheiee-itemfn-fill'-in^temsT^essay^ 



or actual performance of a task.^^If given imme- 
diately after learning sequences, it is a test of 
acquisition; if given considerably later, it is a re- 
tention test: if it requires performance not spe- 
cifically learned during instruction, it is a transfer 
teSt. 

Decision Logic Chart/Table Guide to as- 
sist in the decision-making process. Represents 
the inputs likely to occur for a given situation and 
recommends a course of action or, if appropriate, 
alternative actions. Also called Decision Tatle, 

Discrimination Making different responses 

to the different stimuli. A discrimination requires 



a person to determine the differences among in- 
puts and to respond differently to each. 

Distributed Practice During learning, the 

process of spacing numerous, relatively short, 
practice sessions throughout the learning period.. 
(See also Massed Practice.) 

Duty A large segment of the work done by 

an individual. The itah'cized paragraph headings 
in the duties and responsibilities section of the 
specialty descriptions in AFMs 36-1 and 39-1 set 
forth the major duties performed in each AFSC. 

Enrichment Supplementary^^pfaterial which 

aids the student m progressing through the course 
but is not considered crucial to learning. 

Feedback Information which results from 

or is contingent upon an action. The feedback 
does not necessarily indicate the rightness of 
an action. Rather it relates the results of the 
action from which inferences about correctness 
can be drawa. Feedback may be immediate, as 
when a fuse blows because a lamp was incor- 
rectly wired, or delayed, as when an instructor 
provides a discussion pertaining to an exam taken 
the previous week, or when completed graduate 
evaluation questionnaires are reviewed. (See, also 
Knowledge of Results and Confirmation.) 

Field Test Tryout'of any training course on 

a representative sample of the target population to 
gather data on the effectiveness of instruction in 
regard to error rates, criterion test performance, 
and time to complete the course. 

Full Mission Simulator A device that allows 



simulation of major tasks related to all crcwmem- 
bers for a given aircraft-mission combination. It ^ 
has the capability of simulating environmental 
conditions necessary for mission performance, in- 
cluding, but not limited to, motion and visual 
systems, flight characteristics, full instrumenta- 
tion of sensors necessary to the mission, and simu- 
lation of environmental stimuli for their activa- 
tion. A fully dynamic system. 

Generalization Learning to respond to a 

new stimulus similar, but not identical, to one 
that was present duriftg original learning: for ex- o 
ample, during learning a child calls a beagle and 
spaniel by the term ''dog." A child who has ^en- 
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erdlized would respond dug* when presented 
with hound as a stimulus 

Group«Pacing A procedure in which stu- 
dents progress together toward the same objec- 
tives; often employed where self-pacing is not 
practical for administrative reasons Also called 
Lockstep. (See also Self-Pacing ) 

HardM^are The physical • components of -a 

system (usually electronic or electrical devices) 
which are utilized in educational processes in- 
cluding computers, terminals, audiovisual devices, 
teaching machines, etc. (See also Courseware and 
Software.) 

Input (See Stimulus.) 

Instructional Objective (See Objective ) 

Instructional Media — The means used to pre- 
sent information to the student. 

Instructional System An integrated combi- 
nation of resources (students, instructors, mate- 
rials, equipment, and facilities), techniques, and 
procedures performing efficiently the functions re- 
quired to^chieve specified learning objectives. 

Instructional System Development (ISD) 



A deliberate and orderly process for planning and 
developing instructional programs which ensure 
that personnel are taught the knowledges, skills, 
and attitudes essential for successful job perform- 
ance. Depends on a description and analysis of 
the tasks necessary for performing (he job, ob- 
jectives«» and'tests clearlyi stated before instruction 
begins, evaluation procedures to determine 
whether or not the objectives have been reached, 
aiTd~Tninhod<r fT>Tn'evTstngrth(r7)racess~1^ 
empirical data. ^ j 

Instructor Guide A publication designed to 

provide the administrator of instructional mate- 
rials with information about the objectives of the 
materials, the procedures involved in their de- 
velopment, suggestions for their optimal use, and 
descriptions of what might be expected from the 
materials based on their previous effectiveness. 



Job- 



-The composite of duties and tasks at- 



Jiially performed by an individual. 

Job Inventory An instrument used for con- 
ducting an occupational survey. It consists of items 



for identification and background information, and 
a list of appropriate duty and task statement. 'The 
job inventory does not include standards of per- 
formance for the duties and tasks listed. 

Job Performance Aid (JP.A) A device, 

book, chart, or other reference which facilitates 
the job performance by reducing the amount of 
information the human performer must recall or 
retain in order to successfully carry out a task. The 
guidelines on a movie projector showing the path 
for threading the film is a job performance aid for 
the projectionist. 

Job Performance Requirements (JPR) The 

tasks required of the human component of a sys- 
tem, the conditions under which these tasks must 
be performed, and the quality standards for ac- 
ceptable performance. JPRs describe what people 
must do to perform their jobs. 

Knowledge of Results-^ — r-A report to the stu- 
dent on the correctness of the response It may 
be a verbal report of right or wrong or a display 
(verbal or visual) of the correct response. (See 
also Confirmation and Feedback.) 

Knowledges Knowledges are not directly 

observable. They involve the use of mental proc- 
esses which enable a person to recall facts, iden- 
tify concepts, apply rules or principles, solve 
problems, think creatively, etc. A person mani- 
fests knowledge through performing associated 
overt activities. (See also Skills.) 

Learner-Centered Instruction (LCI) An in- 
structional process in which the content is deter- 
_ jnineibx t h_e_l e arne rs ' n e eds, the j ns t r uc t i on a 1 m a- 
terials are geared to the learners' abilities, and 
instructional design makes the learners active 
Ifrticipants. The instructional system development 
process produces learner-centered instruction. 

Learner-Controlled Instruction An instruc- 
tional environment in which the student can 
choose from a variety of instructional options for 
achievement of the terminal objectives. Students 
can vary their rate of learning, the media used, etc. 

Learning A change in the behavior of the 

learner as a result of experience. - The behavior 
can be physical and overt, or it can be intellectual 
or atti^dinal. 
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Learning Center A learning environment 

which has been specifically developed to foster in- 
dividualized instruction and which emphasizes em- 
ployment of media to augment textbooks and 
manuals. 

Massed Practice During learning, the proc- 
ess of providing all practice sessions at a specific 
point in the learning period (usually at the end' 
of instruction). This is generally considered in- 
ferior to distributed practice. (See also Distributed 
Practice.) 

Measurement, Criterion- Referenced The 

process of determining, as objectively as possible, 
a student's achievement in relation to a fixed 
standard which is based on criterion objectives. 

Measurement, Norm-Referenced The proc- 
ess of determining a student's achievement in re- 
lation to other students. Grading ''on the curve" 
involves norm-reference measurement since an in- 
dividual's position on the curve (grade) depends 
on the performance of other students. Cieneraliy, 
norm-referencec! measurement is not appropriate 
in ISD. 

Medium (plural,Media) A means of effect- 
ing or conveying something Medium is a general 
term roughly comparable in many ways with tool, 
instrument, vehicle, means, etc. 

ModullEir Scheduling \ course is divided 

into small units of instruction called modules 
Each module supports one or more training ob- 
jeelives. Students are pretested and counseled on 
objectives to determine which modules of instruc- 
tion they require Students receive only those 
modules that pretesting and counseling indicate 
they need. 

Multimedia Approach The correlatej^l use 

of more than one type of^ instructional medium 
as a vehicle for presenting the instructional ma- 
terials. Characteristically, an instructional pack- 
age which employs a multimedia approach may 
use textbooks, films, slides, etc., to present various 
segments of the entire package 

Muintrnck Course A course which employs 

more than one track or channel of instruction 
Course goals are the same on all channels, but 
course content, degree of instruction, and presen- 
tation all vary to accommodate students of dir- 



ferent aptitudes and levels of previously^ acquired 
skills and knowledges. 

Objective Objectives specify precisely what 

behavior is to be exhibited, the conditions under 
which the behavior will be accomplished, and the/ 
minimum standard of acceptable performance. / 

Occupational Survey The Air Force pro- 
cedure for the identification of the duties and tasks 
which comprise one or more shred-outs, prefixes, 
specialties; career field ladders, or utilization 
fields, and for the collection, collation, and analy- 
sis of information concerning such duties and 
tasks. 

Part-Task Practice An exercise, performed 

with or without a training device, which allows 
students to practice some portion of a task or set 
of tasks. (See also Whole-Task Practice ) 

Part-Task Trainer A system that provides 

dynamic simulation of some subset of mission re- 
quirements The subset may be defined in terms 
of crew positions and 'or mission segments It 
will include only those capabilities necessary for 
dynamic simulation of the tasks for which it, -is 
designed Instrument flight simulators and aerial 
refueling part-task trainers both fit this definition. 

Peer Training A method of instruction in 

which a student who has completed training acts 
as an instructor to another student in the skill or 
process t(X|^ learned This procedure continues 
with cachT^inee becoming an instructor- for the 
next trainee. 

Perception The process of information ex- 
traction The process by which a student receives 
or extracts information from the environment 
through experiences and assimilates this data as 
facts (sight, sound, feel, taste, smell) 

Performance The carrying out of an act to 

completion, ^actual accomplishment of a task to 
some preset standard of completeness and ac- 
curacy. 

Performance Measurement The process of 

'defermining if the student's performance on a 
given task reaches the standard for that specific 
tasks 

Plan of Instruction (POI) A qualitative 

course control document designed for use pri- 
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maril) within an Air Furcc school tor cuur^e 
planning, organization, and operation Generall). 
for every block of instruction within a course, 
criterion objectives, duration of instruction, and 
support materials/ guidance factors are listed (also 
called a Syllabus). 

Position The duties and tasks established 

as the work requirement for one individual \ 
position exists whether occupied or vacant 

Posttest A test given to a student upon 

completion of a course of instruction to measure 
learning achieved 

Pretest A test given to a student prior to 

entry into a course or unit of instruction to de- 
termine the technical skills^ and knowledges (en-' 
tering behavior) he possesses in a given subject. 
Can be used to identify portions of the instruc- 
tion the student can bypass. 

Procedures Trainer A nondynamic system ' 

that allows procedural training to be accom- 
plished It could take the form of a mockup or 
a cockpit procedures trainer. 

Programmed Instruction A student-cen- 
tered method of instruction which presents the in- 
formation in planned steps or increments, with 
the appropriate response immediately following 
each step. The student is guided step-by-step to 
the successful completion of the assigned task or 
training exercise- 
Programmed Instructional Material In- 
structional material, such as texts, tapes, films and 
filmstrips, slides, scripts for live presentations, etc , 
prepared specifically to employ techniques of pro- 
gramming. 

Response — ^Any activity which is induced 
by a stimulus. In instruction, it designates a wide 
variety of behaviors which may involve a single 
word,^ selection among alternatives (multiple 
choice), the solutioh of a complex system, the 
manipulation of buttons or keys, etc.. 

Response, Covert An internalized response 

which the student presumably makes but' which is 
neither recorded nor otherwise available to an 
observer (for example, a student "thinlts'' a re- 
sponse). 



Response Mode Ihc manner in which a 

student responds, for example, writing a sentence, 
selecting an answer from a group of choices, re- 
pairing a piece of equipment, etc. 

Response, Overt A student's oral, written, 

or manipulative act which is, or can be, recorded 
by an observer. 

Self-Pacing A procedure in which students 

can progress through an instructional program 
at their own rate. (Sec also Group-Pacing.) 

Shaping A technique which reinforces suc- 
cessive approximations, starting with behavior 
that is already present (for example, questions the 
student can answer already) Gradually, more 
difficult material is presented and more sophisti- 
cated answers are required. 

Simulation A technique whereb> "job- 

worW phenomena are mimicked, in an often low 
fidelity situation, m which costs may be reduced, 
potential dangers eliminated, and time com- 
pressed The simulation may focus on a small 
subset of the features of the actual job-world 
situation. 

Simulator A generic term including full 

mission simulators, part-task trainers, procedures 
trainers, etc Simulator and simulator training 
device ma> be used interchangeably 

Skills Skills involve' physical or manipula- 
tive activities. They often require knowledges for 
their execution. All skills are actions having spe- 
cial requirements fur speed, accurac), or coordi- 
nation (See also Knowledges.) 

Software The programs and routines used 

to extend the capability of automatic data process- 
ing equipment (See also Courseware and Hard- 
ware.) 

Stimulus The event, situation, condition, 

"signal, or cue to which a response must be made 

Subject Matter Specialist A person who has 

thorough knowledge of the material being pro- 
grammed. He acts as advisor and critic concern- 
ing subject matter during the production of pro- 
grammed materials. 

Subsystem A major functional subassembly 

or grouping of items or equipment which is es- 
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sential to the uperational LOinpletcncs:^ ut a 
system. 

Survey Test The i>urve> test is a criterion- 
referenced test used prior to the development of 
an instructional system. It is administered to a 
sample of prospective students to determine what 
skills and knowledges should be put intu the course 
of mstruction. 

System The composite ot equipment, skills, 

techniques (includes all related facilities, equip- 
ment, materials, services, and personnel) that is 
capable of performing and or supporting an op- 
erational role 

Systems Approach to Training (See Instruc- 

' tional Synem Development (ISD) ) 

Target Population The persons tor whom 

the instructional or traming materials are de- 
signed Samples from this population are used 
in evaluating instructional materials durmg their 
development 

Task A unit of work activity or operation 

which forms a significant part ofV duty. It con- 
stitutes a logical and necessary step in a perform- 
ance, and usually has a logical beginning and end. 
(Task statements may be found under duties listed 
in A^FMs 36-1 and 39-1 ) 

Task Analysis As used in the handbook, 

includes the process of describing job tasks in 
terms of JPRs and the process of analyzing these 
JPRs to determine TRs (See also Job Perform- 
ance Requirements f JPR) and Training Require- 
ments (TRs) ) 



Threshold Knowledge Test (TK r)-(See Sur- 
vey Test.) 

Trainer A job performance oriented device 

designed to simulate conditions inherent in the 
equipment which it represents 

Training Aid An> item which is developed 

and or procured with the primary intent that it 
shall assist m training and the process of learn-, 
mg. 

Training Requirements (TRs) — TRs are those 
skills and knowledges which arc required for 
satisfying the job performance requirements, and 
not already in the students' incoming repertoire. 

Transfer of Training Ability of the learner 

to apply old (familiar) concepts to new situations. 
Transfer of training is most effective when the 
learning situation is so organized as to facilitate 
generalization and the recognition of relation- 
ships 

Validation The process of developmental 

testing, field testing, and revision of instruction 
to be certain that the instructional intent is 
achieved. The system is developed unit by unit 
and tested (or validated) on the basis of the 
objectives prepared for each unit. Validation 
allows instructional designers to guarantee speci- 
fied results. 

Whole-Task Practice An exercise, per- 
formed with or Without a training device, which 
allows students to practice an entire task at one 
time. (See also Part-Task Practice.) 
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Annotated Bibliography 



There are nurrrcruus publications, both govern- 
mental and commercial, which will tud writers in 
developing and editing instructional materials, The 
following paragraphs procnt recommendations of 
several publications which c^in he useful in these 
endeavors. 

GOVERNMENTAL. PUBLIC ATIONS 

"Air Force Publication Mamij^ement Program. 
AFM 5-1 Washington, DC 1 October 1968 
This manual, especiall) in chapters 7, 8, 9, and 
15, pertains to all who prepare training liter- 
ature. Th^se chapters present the steps in devel- 
oping and issuing a publication, how to prepare, 
review, and edit manuscripts, and how to initiate 
changes to publications 

Guide for Air Force Writint,' .\¥P 13-2 Wash- 
ington. DC 1 Nov 1973, This pamphlet pro- 
vides one standard writini: guide for all Air 
Force personnel who write or approve the writ- 
ing of others It covers the principles of clear, 
logical thinking and writing, with specific sug- 
gestions for applying these principles to Air 
Force written communications One entire chap- 
ter IS devoted to the preparation of publications. 
NOTE The Directorate of 'Administration at 
each Air Force Base has the multimedia (slides, 
audiotape, student exercises, and instructor) Air 
Force Writing Course which correlates with 
AFP 13-2 This course was prepared through 
application of the ISD process 

COMMERCTAL PUBLICATIONS 

A Program for Effectne Writing Robert L. 
Shurter and James M. Reid, Jr. New York, 
Appleton-Century-Crofts, Tnc , 1966 This pro- 
grammed text presents composition in terms of 
clarity, conciseness, force, appropriateness, and' 
organization 



Harhrace College Hamlhuuk John C Hodges and 
others, New York. Harcourt, Brace and World. 
Inc., 1972. This book is one of the more conven- 
ient grammar and writing references. The front 
and back covers of the book contain infor- 
mation to identify the numerically-designated 
sections 

How to Write, Speak, and Think More Effectively 
Rudolf Flesch New York Harper and Row 
Publishers, Inc, 1960 It included material from 
five other books by Rudolf Flesch' The Art oi 
Plain Talk, 1 he Art of Readable Writing, J he 
Art of Clear Thinking, How to Test Readabilitw 
and A New Wa\ to Better English. The author's 

^ well-known Reading-Ease Chart is also in- 
cluded, t 

The Jechnique of C lear Writing Robert Gunning 
New York McGraw-Hill Book Company. 
1968 The author applies the principles of read- 
able writing to newspaper, business, legal, and 
technical writing The book contains much 
useful information about words and their use. 

Writer^ Guide and Index to English Porter G. 
Perrin and Wilma R Ebbitt. /GtenvTew, IL, 
Scott, Foresman and Company^ 1972, This is 
a reference and guidebook that presents a real- 
istic description of current/American English 
usage and style. It can \]t considered a com- 
plete course in EngliTJh'^mposition as well as 
a handy reference manual. 

English Review Manual. James A Gowen New 
York McGraw-Hill Book Company, 1970. A 
programmed text which stres-NCs a review of 
grammar and punctuation. The text also deals 
with the principles of written English, such as 
basic sentence parts, major parts of speech, 
and phrasing 
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Weisgerber. -Robeit A. Ferspettives hi Indivulualized Learnm\i 
Itaska, IL. R E. Peacock Publishers. Inc. 1971. 

Wilkinson, Gene L. "Cost Evaluation ot Instructional Stiategies," 
A V Communication Revtew. Vol. 21. No 1. Spring 1973. 

VOTE. DOD activities can purchase documents identified with 
an ''AD" number from Defense Documentation Center, Cameron 
Station, Alexandria. VA 22314 Others can purchase th|f?e docu- 
ments from US Dept of CrTmmerce, National Technical Informa- 
tion Service, Springfield. VA 22151. 
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VWM ShRlES 

The Air Puree is developing a 16mni filnPsenes on 'Instructional Technology The State-of-the-Art." 
The first five films in this series are* 



TF67()(), ''Instructional Systems Development. The Process" (27 min) 

TF670I, ''Criterion Objectives The Key 4o Success" (16 min) 

TF67()2, "CritcriQn-Reterene*cd Testing** (19 min) 

rF6703, "Indi<^idiializcd Instruction'' 19 mm) 



'^ 



TF67()4. 'The Changing Role of the Instructor*' ( I F mm) 
\ * 

These films are available fur tree luan from audiovisual libraries located at most Air Furce bases. Or- 
ganizations* nut withm convenient distance of an Air^orce bj^e for local over-the-counter film service 
should contcUt TSAF Central Audiovisual Librar>. Audiovisual Service, Norton AFB, CA 92409. for 
ordering instructions and request forms. r 
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Instructions for Completing the I5D Status 

Report 

ISD Status Report, RC'S HAP-DPP(SA )73()3 Outline ot Data Required 

The following is an guthne <rf cxemplarv data whith the report is intended lo provide. The report eon- 
tent md>/var^'' somewhat" for different commands (Coordinate specific report content for each com- 
mand with AF/DPPTB) 

A~ISD Projects: ' 

1 Course — Identify all projects listed b> course number aixl title ' 
. 2 Start Date—hKUCiiic the date the ISD effoit begun 
B— Percent Complete (Steps): 

1: Indicate the percentage of completion qt each applicable step ot the ISD model. 

2. If the project was terminated at anv point prior to implenientatiuii. indicate tl\e percentage of 
coiDpletion that was attained in each' applicable step and brielK state the reasod for termination in 
the remarks section 

C—lmplementation Date: 

Indicate the date the course was implemented oi lange ot time (beginninLi and ending dales of im- 
plementation) over which implementation will occur 

Course Length: V 

Give information indicating the etfeet on eoLirse Jength For example, the beture and aher course 
lengths, where the ' before" figure is the iotal number of training da\s ^md/ or H} ing hours as applicable, 
of the project indicated in section A prior to application of ISD. The ^\ifter" figure will reflect any« 
change in hours after implementation of the project If the proieet is terminated, enter final course 
length. . " . 

E— Student Flow: 

Provrde information which quantitativeU dcstribes the student How. For example, show student IIonV 
for the project 'course identified in Section A Provide in terms of student How for a specific time 
period (2()()/yr. 3()/vvk) using the most current intormation available. 

Remarks: / 

Use the remarks section to provide amplifiijation or elantication-^t an) of the precedmg sections 
or any additional information considered releviuit 

V 



